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training workshops and three times during the year; reports and 
meetings of evaluation coordinators (one in each of the 23 
participating schools) ; and standardized student achievement tests- 
Estimates of pupil progress in IMS are extrapolated from the overall 
test results, and results in four selected schools are analyzed in 
more detail. Observations on the implementation and usage of the IMS 
materials are drawn from the verbal reports. The evaluation shows 
that IMS is meeting most of its goals, but several recommendations 
for specific improvements and extensions are derived. (MM) 
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Chapter I - Introduction 
Description of IMS 






The Individualized Mathematics System i(IMS) is a low- 
cost, brightly illustrated mathematics curriculum for 
grades 1-6. It is designed to provide maximum creativity 
and flexibility for teachers and pupils, and should not 
penalize pupils if they are not proficient in reading. 
Diagnostic tests enable the teacher to prescribe the 
specific topic, level and learning style appropriate 
for each pupil. Other tests incorporated into the curric- 
ulum give the pupil immediate feedback on his work and 
are usually scored by the pupil himself. 

IMS has been developed over the past three years by 
the Center for Individualized Instructional Systems in 
Durham, North Carolina. The Center is an affiliate of 
the National Laboratory for Higher Education, formerly 
known as the Regional Education Laboratory for the Carolines 
and Virginia. During the 1970-71 school year, IMS was 
used on an experimental basis by more than 10,000 pupils 
in 37 schools in North Carolina, South Carolina, Virginia 
and Florida. Previously, major sections of the curriculum 
were tested in eight schools in North Carolina and 
Virginia. A revised version of IMS will be field tested 
by approximately 135 schools across the nation during 
the 1971-72 school year. 

ERIC 
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IMS covers 10 topics: numeration, addition, subtraction, 
multiplication, division, fractions, mixed operations, 
money, time and measurement. Within each topic, there 
is a carefully arrange*.d sequence of skills to be learned. 

The skills are organized into nine levels of difficulty, 
and each skill, or "behavioral objective," is incorpor- 
ated into a separate skill folder. There are 376 skills 
in the IMS curriculum. Figure 1 shows the number of skills 
in each IMS topic at each of the nine levels. 

IMS is not a workbook or a textbook. It is a series 
of about 7,500 brightly colored, deftly illustrated 8.5" x 11 
pages. Each page is laminated in plastic. Pupils mark 
on the plastic pages with special pencils and pens and, 
when they have completed their work and scored it, wipe 
the pages clean with a paper towel and return them to 
a mobile storage cart. The pages can be used over and 
over again, and are expected to last at least five years. 
Mobile storage carts house the entire curriculum, the 
various tests and the manipulative devices. The carts 
eliminate the need for storage space, simplify filing 
techniques and enable IMS to be rolled from classroom 
to classroom. 

The skill folder, the "atom" of the IMS molecule, 
contains from four to more than 20 pages. The skill 
folders provide for various types of learning styles, 
from manipulation of concrete materials such as tokens 
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and counters to increasingly difficult perceptual and abstract 
styles. The number of written directions especially in the lower 
levels, is minimal so pupils with limited reading ability can 
successfully use the materials. Figures 2 through 4 are pages 
from skill folders covering the topic, Numeration, at different 
levels of difficulty. 

The use of a large number of manipulative devices and educa- 
tional "toys," such as balance scales, centimeter rods and measur- 
ing cups, has been incorporated into the curriculum. A list of 
the minimum collection of such items recommended for use with IMS 
(Appendix I) was sent to all schools. The schools' use of this 
information and materials is discussed in Chapter V, pages 80-81. 
There are also pages of special activities for pupils to use 
independently, and some for teachers to use with small groups. 
Figures 5 and 6 are teacher and student activity pages from 
Addition skill folders. Each folder contains two checkup 
tests which the pupil uses to find out if he has mastered the 
skill. Figure 7 is a checkup test from a Measurement skill folder. 

The entire system is color-coded, so pupils can easily locate 
the materials prescribed for them. All skill folders and pages 
for Numeration, for example, are pink; materials for Fractions 
are yellow, and so on. In addition to helping the pupil find the 
materials he needs, the color-coding provides an additional ad- 
vantage for individualization. The color identifies only the 
topic the pupil is working on, not the level at which he is working. 

Imaginative drawings, color and familiar frames of 

reference are introduced into the curriculum to make it more 

o 
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attractive and relevant to pupils. The illustrations 
include a variety of benign monsters, "grape man," and 
a whimsical but improbable being simply known as "the blob." 

As one fifth grader at Thomas Jefferson Elementary School 
in Falls Church, Virginia, wrote: "I like your cartoons. 

They are very tasteful. There are a lot of tasteless car- 
toons around, but yours are very nice." Another classmate 
added: "We all like the cartoons in the math folders. 

They help you understand the directions better. They also 
help you like to work in math more, even though we didn't 
like math that much . " 

The IMS curriculum moves the pupil through a series of 
small steps toward the accomplishment of specific, measurable 
objectives. It represents an uninterrupted sequence into 
which the child can enter at any point. Testing in IMS 
is used to aid the pupil, not to categorize him. The place- 
ment test determines his starting point (topic and level of 
difficulty) , and a pretest, taken before he begins work, 
determines the particular skills in Which he needs instruction 
within that topic. Figures 8 and 9 show the placement test 
and corresponding pretest for Level II Time. 

For example, the placement test might indicate that 
the pupil should begin work in Measurement, Level IV. There 
are four skills in Measurement, Level IV, and the pretest 
might indicate that he has already mastered Skill 1 and 2. 
Therefore, his prescription (teachers write prescriptions 



for lower levels of t ie c .rriculum; pupils often write their 
own prescriptions for advanced levels) will direct him to 
the folders for skills 3 and 4. 

The checkup tests, generally scored by the pupil, 
tell him whether or not he has mastered the skill in the fold- 
er. If he does not attain mastery score on the first checkup 
test, he can take the second after he does further work 
in the folder or has some help from his teacher. When all 
skill folders in a unit heive been mastered, the pupil takes 
a unit posttest. The posttest indicates whether the pupil 
is ready to go on to the next prescription, or whether 
additional work is needed. It is essentially an alternate 
form of the pretest. 

Frequent testing is an integral part of IMS. Since 
it occurs at each small step along the way, it becomes a 
means of proving success rather than a psychological road- 
block. The pupil soon learns that the tests are not there 
to punish him but tb help him. As one fourth grader wrote, 

"I like IMS because the teacher trusts us to score out own 
test. It makes me feel like a teacher myself." 

How Does IMS Work? 

At the beginning of the school year, several hours of 
class time are devoted to pupil orientation. The students 
learn how to find the materials they need, how to score them- 
selves (in the lower grades, teachers and teacher aides 
help pupils with scoring) and how to record their scores. 
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They also practice wiping pages clean and returning thorn 
to the storage cart. Each student makes a simple construction- 
paper flag (often inscribed "Help!" or "Not Together!") 
which he props up on his desk when he needs help. 

Then the class takes a placement test, which determines 
for each student "where he is" in the system regarding the 
ten topics and' nine levels of difficulty. From the placement 
test, the teacher makes up a "pupil profile chart" for each child. 
Figure 10 shows the profile chart for a student who has 
mastered at least Levels I through III in all topics from 
Numeration through Time. He has mastered Levels I and II 
in Measurement. The profile chart indicates he should tackle 
Measurement, Level III, which has five skills of "behavioral 
objectives" in it. 

This student then takes the pretest for Measurement, 

Level III, and the results might indicate he has mastered 
everything but Sk^.11 2. Skill 2's objective calls for the 

j 

pupil to use a rujLer with one-inch divisions to measure 
real objects or pLctures to the nearest inch. The student 
reads his prescription, then finds skill folder 2 in the 
pigeonhole on the cart marked Measurement, Level III. There 
are seven pages in the folder, and the prescription might 

I 

call for work on pages one through four. The student gets 
a ruler from the storage cart with the manipulative devices 
and begins work. 
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NAME GRADE ROOM 










13 



After completing the four pages, the student takes the 
checkup test. He is asked to measure drawings of a pencil, 
a nail, a wrench and a paintbrush. He writes in his answers, 
checks them against the scoring key and records his score. 

He should get four right, which is the mastery level for this 
skill folder. He then erases the pages, returns the skill 
folder to the cart and takes the unit posttest. He should 
achieve mastery level on all five skills of Measurement III, 
all of which are covered on the posttest. 

The whole process begins again. The student goes back 
to the pupil profile to see what he is to do next and takes 
the pretest for the new unit. Then he receives a prescription 
(or writes his own) for pages to teach the first of the skills 
identified by the pretest as needing work. There are always 
stumbling blocks, of course, but the possibilities for solu- 
tions are many. If the student does not show mastery of 
a particular skill on the posttest, or in any checkup test 
along the way, he may simply require further practice. Thus, 
more work pages might be prescribed or a group activity 
might be used to clarify a concept or an advanced pupil might 
be asked to help the student. 

Often the teacher may decide to work with a child indi- 
vidually or in a small group until a given skill is acquired. 
Thus, the classroom may be organized to make use of IMS 
materials in many ways. But the fact remains: each child 

is being taught individually because he is moving at his 
own pace in a program tailored for him. Dennis Nichols, 
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principal of Morehead School in Durham, points out that achieve- 
ment in IMS represents mastery of the material and all the 
requisite skills preceding it. "Under the textbook method," 
he said, "pupils were often going too fast, being exposed 
to material but not really understanding it. Using IMS, they 
stay with the material until they really master it. The 
curriculum gives us a way to discover and remedy deficiencies 
on an individual basis*." 

Because IMS is individualized, pupils do not receive 
standard letter grades measuring their progress against that 
of others in the class. Instead, the IMS report card is a 
variation of the pupil profile chart. Parents receive this 
form, called a "pupil progress report," four times during the 
semester. The color-coded chart tells parents where their 
son or daughter entered the system on the placement test, 
and what progress he has made. 

The role of the teacher in IMS is clearly different from 
that in the traditional math classroom, but certainly no 
less importaht. "IMS provides an opportunity to make maximum 
use of the creativity and flexibility that master teachers 
always have brought to the classroom," says J. W. Knight, 
director of the Center for Individualized Instructional Systems. 
"IMS allows teachers to abandon the lockstep pace of the 
lecture method and become prescribers or managers of learning. 
They spend most of their time working individually with pupils 
or with small groups — which is what they do best." 



Plan for Evaluation 



The funding for the evaluation reported herein was 
not for the purpose of determining the effectiveness of 
IMS as compared to some other approach to teaching mathematics. 
The IMS stage of development for 1970-71 was not sufficiently 
advanced to warrant comparative evaluation. Instead, the 
evaluation had as its goal bringing about changes in IMS in 
order to make it more effective for subsequent use. Some 
results reported in subsequent chapters are the revision of 
about 20% of the tests in the system and the revision of over 
250 teaching pages, which includes complete reorganization of 
four skill folders. In addition, teacher training procedures 
have been revised. Therefore, any comparative results with 
respect to usage of IMS in 1970-71 would be essentially 
invalid, since the system should be improved for future use. 

In addition to effecting changes in IMS the evaluation 
was conducted to determine whether IMS has achieved certain 
goals, either as initially introduced or as a result of 
changes made during the year. Specific goal areas are: 

Curriculum Adequacy . The provision of a comprehen- 
sive set of mathematics objectives suitable for 
a wide spectrum of pupil aptitudes. 

i' . 

/ Materials Effectiveness . The provision of attract- 
ive and effective learning materials and teaching 
aids which incorporate various alternative means 
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of achieving curriculum goals. 

Cost-Effectiveness . Achievement of a low cost 
per pupil compared with other available mathe- 
matics systems with similar structure. Specific 
areas to be considered in this repsect are: 

Actual production costs and adequacy of 
reusable materials. 

The extent to which students can and do assume 
responsibility for operation of the system 
(thus reducing or eliminating the need for 
paraprof essional personnel in the classroom) . 
The cost and effectiveness of teacher train- 
ing required to implement the system. 

Learning Effectiveness. Pupil achievement and 
progress within the system comparable with or 
superior to that obtainable under conventional 
teaching conditions. 



Schools 

To investigate effectiveness of IMS with respect to 
these goal areas 23 elementary schools in the Carolinas 
and Virginia were selected for field testing of IMS. These 
schools cover a wide range of educational situations 
including urban deprived, rural, middle-class-urban and 
upper -middle-class-suburban. Over 5000 students were involved. 

Four of the 23 schools were chosen for collection of 
detailed data on pupil progress. Each of these is quite 
different from the other three , and combining their student 
populations yields a diverse yet reasonably representative 
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sample. At various times, however, it will be necessary to 
relate results to the specific school of origin. For this 
reason, these four schools cire now described in terms of 
their more outstanding characteristics. 



Enterprise Elementary School . This school fills a very 
difficult rol& and does so very well considering the 
circumstances . Virtually the entire student population 
comes from homes where poverty, if not present, is always 
lurking in the background. Parents are mostly mill employees 
and are subject to seasonal and other layoffs. A recent 
survey revealed that parents of only three students 
had ever even attended college. Social conditions are 
often unstable for these families. The restrictions 
of an urban environment under much less than affluent 
circumstances weigh heavily on these families of pre- 
dominantly southern rural extraction. Enterprise is 
about 40% black due to recent desegregation, and the 
changing position of black people with respect to poor 
whites probably heightens social tensions. However, 
there have been no racial disturbances in the area. Only 
a small proportion of the students achieve grade-level 
or higher than grade equivalent scores on standardized 
tests. Teachers report frequent minor (sometimes major) 
behavior problems which they associate with unfortunate 
home conditions for many of the students. 

Academia Elementary School . Only about 10% of the fathers 
of students at this school have eighth grade or lower 
educations. In contrast about 53% of fathers have at 
least some college training and 11% hold doctorates. The 
school is considered "experimental" and has excellent 
facilities and equipment. A nearby university frequently 
provides teaching assistants and trainees and conducts 
special programs. Approximately two- thirds of the 
teachers hold masters degrees. Although a small proportion 
of the students come from "deprived" backgrounds and 
show some school achievement deficiencies, the average 
student scores well above grade level on standardized 
tests. Less than 2% of the students are black. 

Countryside Elementary School . The families of Countryside 
are probably at about the same socio-economic level 
(on the average) as those of students at Enterprise. 

The school's facilities and equipment are not as good as 
those of Enterprise. Nevertheless, Countryside seems 
to take greater advantage of those facilities and materials 
which are available to them. The school serves a rural 
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clientele near an urban center. Most oC the families 
live on small farms that have boon in their families for 
many years. However , this kind of farming has become largely 
uneconomical, and most of the fathers hold blue collar 
jobs in the nearby urban center. Many of the mothers work 
also. While their work is not more stable than that of 
the parents at Enterprise, they have their farms to 
"tide them over." Most raise vegetables and many keep 
some dairy cattle. About 10% of Countryside's students are 
black, the same ratio for the community as a whole. 

Social and economic relationships between blacks and whites 
have matured over many years and there is no sign of tension. 
The average student is approximately at grade level on 
most standardized tests. Teachers report very few dis- 
cipline problems. 

Urbania Elemen tary School . This school is near the edge 
of a large southern metropolitan complex, yet it cannot 
be called "suburban." It is in a neighborhood of modest 
homes, but about 40% of its students are bussed from a 
nearby black neighborhood. Almost three- fourths of the 
black children are from families receiving welfare payments. 
In total, 30% of Urbania' s students represent severe 
poverty conditions . Only about 5% of Urbania' s students 
have fathers whose occupations might be described as 
"professional," loosely defined here as requiring a college 
degree. About 20% are white collar nonprofessional. 
Employment for approximately 50% of the fathers classi- 
fiable as blue collar workers is probably more stable than 
that for fathers in Enterprise. Students at Urbania re- 
present a wide range of academic achievement, but on the 
average they score approximately at grade level on most 
standardized tests. The school facilities are excellent 
and it is in a district which provides extensive support 
in terms of teacher aides, instructional materials and 
supervisory assistance. 

In further discussion, these schools will be referred to as 

the intensive evaluation schools or individually by name. 



Teacher Training 

During the summer of 1970, four workshops were held to 
introduce school personnel to IMS practices and materials . 
One was for principals and supervisory personnel, and three 
were for teachers. Approximately 50 attended each workshop. 



Vt'.'l'.Jl.'UHHilU 






19 



These workshops are considered an integral part of the IMS 
program, and their evaluation influences many outcomes 
reported herein. 

Data Collection 

This aspect of evaluation is extremely diverse, and many 
sets of data will be used for multiple purposes. To organize 
this process for presentation in this section, each major 
area of evaluation is discussed in terms of data collection 
involved. 

Curriculum Adequacy . A specialist in mathematics education, 

Dr. Joseph Scandura, of the University of Pennsylvania, was 
asked to evaluate IMS structure (Behavioral Objectives) 
and materials with respect to suitability for enabling 
students to achieve the stated objectives. In addition, . 
he was asked to evaluate the materials with respect to 
mathematical consistency and correctness and completeness 
of subject matter coverage with respect to preparation for 
topics to be met in later mathematics courses. 

A specialist in learning theory, Dr. Robert Gagn6, of 
Florida State University, was asked to evaluate IMS order 
and method of presentation and Behavioral Objectives from the 
standpoint of appropriate learning theories and child 
development fcesearch. 

Teacher Training . At the end of each workshop, all participants 
responded anonymously to the questionnaire, IMS WORKSHOP 
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EVALUATION FORM (Appendix III). However, the main sources 
of evaluating the effectiveness of teacher training came 
from other sources. Monitor Reports (Appendix IV) 
were prepared by IMS personnel visiting schools at various 
times during the: school year and provided data regarding 
training. Three Teacher Surveys (Appendix V) contained 
questions regarding teachers' later opinions of workshop 
effectiveness . 

Installation and Operation Procedures . A teacher was selected 
from each school to act as an Evaluation Coordinator (EC) . 

EC's were responsible for coordinating the collection of 
pertinent data within their schools and participated in 
modification planning. They arranged to have other teachers 
in their schools furnish information concerning the operation 
of the system. Much of this information was forwarded on 
Incident Report forms (Appendix VI) to the Laboratory on 
a weekly basis. Monthly EC workshops were held to discuss 
the information collected and to aid in analyzing the practicality 
of teacher recommendations. 

The EC's participated in several other phases of data 
collection in addition to that just described. All these 
activities were coordinated through the monthly workshops. 

Based on school visits, IMS personnel reported many 
data regarding installation and operating procedures. These 
observations were usually reported through use of the IMS 
Monitor Report (Appendix IV). 
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System Effectiveness. Teacher evaluation of system effective- 
ness was obtained through use of the three IMS Teacher 
Surveys (Appendix V) . Some of the topics covered are: 

. Student assumption of responsibility for operation 
of the system 

. Adequacy of materials to permit selection of appro- 
priate learning tatek to fit needs of individual 
pupil 

. Reaction of pupils to materials 

. Availability of time to spend with individual pupils 
The Teacher Surveys were administered in December, March 
and May by the EC's. 

To be effective, the system depends heavily on the 
validity of the tests used to govern progress between levels. 
To investigate this question, all posttests completed by 
students at the four intensive evaluation schools were 
forwarded to the Laboratory . Item responses were recorded 
for later analysis, and tests were returned to the students. 

Pupil Achievement and Progress . The Metropolitan Achievement 
Test (MAT) - Arithmetic (grades 1 and 2) and the Iowa Test 
of Basic Skills (ITBS) - Mathematics (grades 3 through 6) 
were used to provide normative data concerning the students 
in the intensive evaluation schools. These tests were 
given in mid-October and again in mid-May. 

EC's obtained from all schools monthly teacher reports 
listing the number of students at each level for each topic 
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of the system. These data bear both on system effectiveness, 
and pupil achievement, since progress from level to level is 
^ an essential feature of the system and is also evidence of 

what skills the students have mastered. 

Costs. Actual production costs were recorded along with costs 
to schools for auxiliary equipment and materials. 

Basis for Effecting Curriculum Changes 

' The evaluation procedures just described are formative in 
that all were conducted in order to determine changes needed in 
various parts of the system. When a broad goal of this kind 
must be accomplished, there are basically two ways to proceed. 
Every element of the system may be tested individually- -each 
teaching page, each activity, etc. Another approach is to seek 
only data which indicate malfunctioning features of the system. 
VJhen resources are limited the second approach is the only 
feasible one. It is not as thorough and may not uncover every 
deficiency. However, due to limited resources, it was 
necessary for this evaluation to adopt the policy of accepting 
as satisfactory elements of the system for which no adverse data 

i 

were noted. Incident Reports (Appendix VI) were especially 
useful in identifying problems with respect to which other data 
collection procedures were not specifically directed. 
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Chapter II 

The Evaluation Coordinator Program 

No small amount of emphasis should be placed on the func- 
tion of the EC program in this evaluation. While their func- 
tion "on paper" was largely one of data collection and 
reporting, a great deal of information passed through them 
to the schools. In most cases, the EC's became the leaders 
and problem solvers for IMS in their schools. They often 
transmitted questions or problems to the IMS staff or the 
evaluation staff and worked with teachers in their schools 
to effect solutions and further explain IMS procedures. In 
many schools, they conducted weekly seminars for teachers 
and aided in presenting IMS to parents and the community. 

It is a firm recommendation of this report that a person 
be appointed in every new IMS school to assist in the manner 
of the EC's. (The Laboratory paid EC's a small monthly 
honorarium for their services, but this payment did not take 
into account all phases of their work as described above.) 

Eight EC meetings were scheduled for the months November 
through June. Two of these were attended by principals 
rather than the EC's because of a need to discuss administrative 
problems connected with installation and operation of IMS 
and for presentation of approaches to informing parents about 
the program. The remainder of this chapter describes the 
content and conclusions of these meetings to the extent that 
they do not appear elsewhere in the report. 
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November - Orientation 

This meeting was devoted to orientation of the EC 
and was held twice, each time for half the EC's. The basic 
plan for evaluation was presented, and data collection 
procedures were explained. The EC's also spent a half 
day with IMS personnel becoming familiar with stocking 
procedures and methods of ordering additional materials. 

December - IMS Practices, Content and Procedures 

Classroom practices and procedures dominated this meeting, 
the first attended by the entire group of EC's. A substantial 
amount of time was spent sharing problems and solutions to 
everyday classroom problems . It was the case that a number 
of teachers using IMS had not attended the summer workshop^. 
This point will be discussed in detail in connection with 
teacher training evaluation results . 

Many of the EC's requested assistance with regard to 
the mathematics underlying certain behavioral objectives. 

It was the concensus that a large proportion of teachers in 
their schools were having difficulty in dealing with the 
following: 

. Explaining why division by zero is undefined. 

. The emphasis .(in IMS) to the effect that zero 
is a number. 

. Prime factorization. 

. Use of least common multiple and greatest common 
factor concepts in finding common denominators. 
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Emphasis (in IMS) on arbitrariness of the convention 
that a 1 x b + c means (a Kb) + c. (Apparently 
some teachers were unaware that this choice is 
arbitrary.) 

Of all these, dealing with zero seemed to caused the most 
problems. Apparently many elementary mathematics programs 
simply avoid consideration of zero. This outcome resulted 
in a recommendation for revision of teacher training procedures 
(see pages 76-781) . 

The- afternoon period of this meeting was devoted to 
touring the recently opened IMS production facility and 
ironing out materials supply problems. 

January - Materials and Logistics Problems 

Difficulties arising from specific skill folders 
were discussed at length. EC's brought lists of these 
problems, and group concensustis were reached regarding their 
extent, severity and the best solutions. Structure for this 
discussion was provided by tabulation of teachers' responses 
to an item on the first Teacher survey, which asked teachers 
to list the pages they felt required revision most urgently. 

The result of this and other efforts to determine revision needs 
is discussed in Chapter VI. 

A second segment of this meeting was devoted to logistics 
problems. EC's presented floor plans of their schools 
t.nd explained the use of IMS materials in these settings. IMS 



personnel collected this information for use in determining 
stocking strategies for 1971-72. 

February - IMS Activities 

Student and teacher activities beyond Level IV had not 
been written as of this time, and the February meeting was 
devoted to a compilation of the activities teachers had de- 
veloped for their own use at these levels. Small presenta- 
tion groups were formed, each led by an IMS representative who 
recorded the proceedings for transmission to writers • 

(Two activity writers were available to lead groups.) 

Another segment of this meeting was devoted to a review 
of evaluation findings to date. 

March - Principals' Meeting 

Principals of all participating schools attended this 
meeting in place of the EC's. After a review of evaluation 
findings to date, the meeting was conducted by IMS personnel, 
who received principals' recommendations and comments regarding 
administrative measures necessary to provide for successful 
introduction of IMS. These recommendations were later passed 
on to principals of schools which began using IMS in September, 1971 

A second phase of the meeting concerned methods of pre- 
senting the IMS program to parents and the community. A major 
outcome of this discussion was the consensus that the majority 
of parents had little basis for distinguishing between the 
mastery achieved using IMS and whatever learning short of 
mastery may be achieved through the usual textbook 
presentation. Concern was expressed. to the effect that parents 
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might feel a student was "short-changed" if in using IMS 
he had mastered only fourth grade topics by the end of the 
sixth grade. Such a student might be of the sort who would 
otherwise get a low "C" in mathematics after a somewhat 
ineffective exposure to the usual sixth grade textbook 
presentation. In response to this area of concern a recommen- 
dation is made to emphasize, in teacher training, the 
importance of communicating to parents the mastery aspect 
of IMS achievement (see pages 72-74) . 

The afternoon period was devoted to tours of the IMS 
production facility and individual conferences with IMS 
personnel regarding materials orders and other school matters. 
A pril - Manipulative M aterials, Presenting IMS to Parents 
and Student Involvement 

To prepare to discuss manipulative materials for student 
use in connection with IMS, EC's brought inventories of 
such materials on hand in their schools and indicated the 
extent to which they were useful for various purposes. 

It was interesting to note that items considered of little 
value in some schools were highly used in others, sometimes 
for different purposes. Some EC's reported that their 
schools had not obtained certain items considered necessary 
for IMS? pan balance scales were most often reported missing. 

A summarization of actual usage reported is in Chapter V, 
pages 80-81. 

The second topic was parent understanding of IMS. Comments 
by principals at the March meeting and questionnaire results 
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had led to the conclusion that parents were generally unaware 
of the fact that progress in IMS is based on mastery of 
specific behavioral objectives. It was explained to EC's 
that careful presentation of this feature is important, since 
otherwise some parents of slower children may not understand 
why they appear to be working on "below-grade level" topics. 
It was suggested that understanding the difference between 
mastery and simple exposure to a topic (often followed by a 
low report card grade) is the key to parent acceptance, both 
of their children's achievement level and continued use of 
IMS. 



Several EC's reported on the success of IMS presentations 
before PTA meetings and other adult groups. The most 
enthusiastic reports came from schools in which students 
had participated in demonstrations of IMS. In some cases a 
demonstration class was run, and in others students explained 
their own work in IMS to their parents after orientation 
sessions by teachers. 

Questionnaire results had shown a wide variation in -the 
extent to which students were given responsibility for per- 
forming various tasks within the system. Several EC's 
explained the bases for decisions relating to student 
involvement in their schools. Consensus was reached 
regarding several points; these outcomes are reported in 
Chapter III. 
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May - First Grade Practices 

Census reports had shown that IMS usage in the first grade 
was less than had been anticipated. Therefore, EC's were 
requested to investigate IMS first grade usage or the lack 
of it in their schools. In response, many were able to 
■ brin 9 first grade teachers or aides to the May meeting. 

The substance of their findings is reported in Chapter V, 

i page 82, where other results bearing on the first grade are 
also reported. 

June - New Principals ' Orientation 

While EC's completed data collection in their schools , 
the principals were invited to a workshop for principals 
whose schools were to enter IMS. in September, 1971. The 
experienced principals participated in discussions and made 
presentations relating their experiences and recommendations 
for introduction of IMS. 
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Chapter III 

Evaluation of Learning Effectiveness 
and Student Progress 

Student Progress Within IMS 

The nine levels of IMS cover approximately the subject 
matter introduced in a typical six-year series of elementary 
school mathematics texts. Therefore, it is of substantial 
interest to users to know the extent to which students may 
be expected to cover this material under individualization. 

Two important factors bear upon this outcome: 
i) Progress in IMS represents mastery. In a typical 
elementary school classroom, the teacher presents a topic 
to the class or groups within the class and gives a test. 

Let us assume that no student "fails" the test. Nevertheless, 
some students do well on tests and others do somewhat poorly. 
The poor performers get lower grades, but the entire group 
or class then moves on to the next topic. In IMS each stu- 
dent stays with a topic until he has mastered it to the 
extent required by the system, usually by performing in 
the 85-100% range on an IMS posttest. Thus it would be 
reasonable to expect that many low ability students will not 
complete the entire IMS curriculum in six years. The IMS-2 
junior high school program, currently under development, calls 
for completion of the first nine levels of IMS in the 
junior high school classroom for students who have not 
mastered the skills therein. 
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ii) Only six of the nine levels of IMiJ were available to 
schools during i970-71. Therefore, estimates of materials 
coverage for six years of elementary sahool are based on 
the assumption that progress (mastery of skills) in the upper 
levels of IMS will occur at the same rate as in the lower. 
This assumption seems justified in view of the fact that, 
throughout the 23 schools using IMS to date, differences in 
rate from grade to grade and from lower to higher levels 
of IMS have been slight. 

This latter result does not mean that IMS progress has been 
uniform in every aspect. Table 1 shows average IMS levels 
for students in the 23 schools participating in the 
evaluation. The basic datum, for which the Table reports 
averages, is each student's average position in IMS. For 
example, suppose a student's placement test locates him at 
Level V in three of the ten areas of IMS and at Level VI in 
the remaining seven. Then his average starting point in 
IMS is Level 5.7. If at the end of the school year he is 
ready to start Level VII in five areas and Level VIII in 
five areas, his ending point in IMS is Level 7.5. These 
average levels are the data for which Table 1 reports means 
by school, and in some cases groups within schools. The 
time covered by the data obtained from schools is only 
two-thirds of a school year and less for some schools or 
groups within schools. Therefore, to estimate gain within 
IMS for an entire school year each mean in column five of 
the table was multiplied by the inverse of the proportion of 
the school year covered by the data. These results are 
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shown in column six. An estimate of the average final pos- 
ition in IMS at the end of grade six was obtained by multiply- 
ing the result in column six by the average number of years 
remaining in elementary school for the students of the subsample 
and adding this product to the mean in column four. The result 
is shown in column seven. 

Three groups are omitted from Table 1, specifically, 

40 second grade students at Lawsonville Avenue, 74 third 
grade students at Chapin and 154 fourth grade students at 
Taylor and Willow Drive. Data for these groups were inconsistent, 
and the sources of error could not be determined. 

Column 7 of Table 1 requires substantial interpretation. 

What does it mean to say that the average student's estimated 
IMS level at the end of sixth grade is 9.38? Of course, it means 
that the average student will complete about four Level IX 
topics and leave six for junior high school. (See explanation 
on page 31.) Half the students will accomplish more than this 
and half less. However, the spread of these final average IMS 
levels attained by students is also of interest. 

Data from the intensive evaluation schools suggest that 
the standard deviation of the final average IMS level scores 
is about 1.0 for groups of students scoring for the most: part 
no more than a half year below grade level on standardized 
mathematics tests. For students more than a half year 
below grade level, the standard deviation is greater, 
approximately 1.5. The distributions involved are essentially 
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symmetrical except for those containing substantial numbers 
of students whose progross in IMS was curtailed due to 
absence of Levels VII, VIII and IX. (In total, there were 
relatively few such students? most fifth and sixth grade 
classes in the evaluation contained students whose progress 
in mathematics had been somewhat retarded. ) 

The smaller estimated standard deviation (1.0) should 
be applied to schools with higher estimated final average 
IMS levels. With a few possible exceptions, the schools with 
final estimated average IMS levels above 7.0 may be character- 
ized as representing the kinds of educational situations in 
which most students are at least near-grade level on standard- 
ized test. The higher (1.5) standard deviation may be applied 
to the remaining schools. 

Application of these standard deviations may be illus- 
trated for Dunbar and Lemon Road (see Table 1) . Since at 
least 90% of cases may be expected to occur within two 
standard deviations on either side of the mean, Dunbar's 
students may be expected to have final IMS averages ranging 
from 2.96 to 8.96 (mean**5.96, standard deviation* 1. 5) . 
Therefore, only a small scattering of Dunbar students may 
be expected to reach Level IX, with even fewer finishing it. 

For Lemon Road two standard deviations (1,0) on either 
side of the mean yields a range of 6.37 to 10.37. Of course, 
IMS covers only through Level IX (up to 10.0) . Therefore, 
a small percentage of Lemon Road students should complete 
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all nine levels of IMS. If the distribution of final average 
IMS levels is approximately normal, this proportion would be 
about 5%. By the same process, about 30% of students at 
Waddell might be expected to complete IMS before the end of 
sixth grade. 

It must be emphasized strongly at this point that the 
above results are estimates only and probably very rough 
ones at that. Several sources of error intrude in addition 
to the absence of Levels VII, VIII and IX. For example, Classes 
starting later in the school year usually made faster 
progress than those starting earlier, probably due to 
profiting from the experience of others. Also, for a few 
schools, progress seems somewhat slower them might have been 
expected when compared with that for similar schools. 

Schools with most IMS students in lower grades appear to have 
systematically higher estimated average final IMS levels. 

This situation seems not to be due to younger groups 
covering more material. Instead, older groups seem to have 
had surprisingly low initial placement in IMS, not all of 
which may be overcome by the end of sixth grade. (This 
outcome is discussed in great detail in Chapter IV.) 

All these things considered, it would appear that the 
estimates for the final average IMS level may be low. Neverthe- 
less, one outcome appears quite definite. A large proportion 
of students cannot complete the first nine levels of IMS during 
the six years of elementary school. For some subpopulations, 
it appears that virtually none will finish. For others, as 
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many as half may finish, but a more conservative estimate 
is 5% to 30% for fairly ordinary elementary schools. 

This outcome illuminates one of the greatest strengths 
of IMS. It gives clear information as to exactly what kind 
of achievement is going on. To say that the average 
graduating sixth grader in school X has a grade equivalent 
score of 6.9 on a standardized test suggests that such a 
student is performing adequately. Yet it is widely acknowledged 
that the mathematical acumen of the average student leaving 
sixth grade is very meager. IMS gives a much more accurate 
picture of what a student at any grade level can actually do 
in mathematics. 

A hypothetical average sixth grader leaving school X 
needs to get only slightly more than half the problems right 
on a standardized test in order to be "at grade- level" 
at the end of the sixth grade. This is not the kind of per- 
formance built into IMS. Further, it raises anew the question 
of the advisability of exposing a large proportion of elemen- 
tary school children to mathematics topics they evidently 
cannot master at the time. The extent of this problem 
becomes more apparent when we stop to consider the fact 
that the half of the students who score "below-grade- level" 
for the most part get less than half the problems right on 
a standardized test. In IMS there is reasonable assurance 
that every student has a genuine grasp of all areas he 
has covered. 

o 
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One further note regarding the results of Table 1 
should be considered. With a few exceptions, results 
reported there cover several grades within a school. This 
kind of reporting was necessary in order to keep data collec- 
tion and reporting activities within the resource limits of 
the evaluation. In contrast, at the four intensive evaluation 
schools, records were maintained on individual students. It 
is largely these data that led to the earlier conclusion 
that progress rates within IMS are reasonably uniform from 
level to level and from grade to grade within schools. 

The real interest with respect to these data, however, lies 
in their relations to standardized test scores. These 
results are covered in Chapter IV. 

System Test Results 

A substantial analysis procedure was undertaken with 
respect to the posttests within IMS. Results obtained 
bear on several aspects of evaluation. The psychometric 
properties of the tests themselves are important, since 
reliable and valid scores are a prerequisite for using test 
results to evaluate other features of the system. If 
reliable and valid, test results may be used to infer the 
degree of success of the materials of the system in fostering 
the achievement of behavioral objectives. In addition, the 
posttests regulate progress through the system. Therefore, 
the meaningfulness of the results presented earlier in this 
chapter is dependent on evaluation of system posttest results. 
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It is not possible to summarize precisely the outcomes 
of this phase of evaluation in a few words. In general the 
results are quite favorable, but certain specific and 
general weaknesses within IMS become apparent upon examina- 
tion of test analyses. The remainder of this section explains 
the methods for analysis and lists major findings. 

Method of Analysis 

From the four intensive evaluation schools, all posttests 
taken by students were collected. From these a sample was 
drawn consisting of approximately 100 students ' responses to 
each of the 60 posttests in use during the year (Level VI I -IX 
not available) . For some posttests fewer than 100 students 
responded. Results are reported hereafter only for a sample 
of 20 or more. 

Each posttest item was assigned a score of 1 if correct 
and 0 if incorrect. The means for these item scores (across 
students) are, therefore, an estimate of the proportion 
answering each item correctly in the larger population of 
all students using IMS. If the proportion of subtest items 
a student must answer correctly to master a skill is, for 
example, 85%, and only 60% of students answering a given 
item in the subtest get it right, something is probably wrong. 
The item may be inappropriate or misleading. On the other 
hand, the students may not have learned what the item 
legitimately tests. Often inspection of the offending item 
and the related learning materials yields an obvious conclusion. 
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The identification of these test items of obvious poor 
quality or obviously inadequate teaching materials based 
on inspection of difficulty levels of items is a major 
contribution of the formative evaluation of IMS. Probably no 
other method could have accomplished as much with the same 
expenditure of resources. 

For some items the proportion of correct responses may 
be "borderline" or there may be no obvious explanation for 
a very low proportion correct. To investigate these cases 
further, the item scores for each student were matched with 
three other scores, his grade-equi valent score (GE) on the 
ITBS or MAT, his age in months at the time of the standardized 
test and the quotient, GE divided by age. Then for each 
item three correlation coefficients (biserial) were computed — 
item scores successively with GE, age and GE/age. This last 
measure may be considered a measure of ability, since 
younger students with higher GE scores will have high 
GE/age ratios. Older children with low GE scores will have 
proportionally much lower GE/age ratios. 

In a successful situation, all three of these coefficients 
should be near zero, within the range of sampling error. 

That is, a student's score on an external measure should not 
influence the success he has after completing an IMS skill 
folder (assuming proper placement) . A student's age 
should not be a handicap or an advantage, since individual- 
ization arranges for some students to complete units 
while much younger than others. Finally, ability as represented 
by GE/age should not be a factor in IMS success on a given posttest. 
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The following section is o summary of findings based on 
the above statistics. This summary includes some findings 
regarding "extension items." These are posttent items that 
were judged to be more difficult or involved than items which 
might be used to test a specific skill. In some cases, they 
are simple problems but require more than one skill for solution. 
They appear at the end of most postbests and are not required 
for skill mastery. They are included for several purposes: 
to add interest for bright students, to give practice with 
fairly conventional problems not otherwise covered and to 
provide research clues for further curriculum development. 

In addition to item analysis outcomes, a table has been 
included which summarizes the results of the biserial correla- 
tions of item scores with GE, age, and GE/age. (See Tattle 2) 

I tons whose scores were found to correlate significantly with 
one or more of these variables were reconsidered and appropriate 
revisions were made whenever possible. 

Level I - Only Multiplication I and Division I posttests 
were available in sufficient quantity to warrant analysis. 

Other data indicated serious need for revision of Skill 2 
of Multiplication I and test results confirmed this need. 

(See Chapter VI . ) Performance on all other areas of the 
Multiplication I and Division I posttests indicated excellent 
achievement, with good difficulty levels and low correlations 
with criteria. High achievement on items 16 through 18 of 
the Multiplication I posttest indicates that children taking 
the tests were able to cope x*ith commutativity of 
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TABLE 2 



Biserial Correlations of Test Item Scores 
With Three Variables 



Significant Correlation/Total Correlations 



Level I 


5/18 


5/18 


0/18 


10/18 




(27%) 


(27%) 


(0%) 


(19%) 


Level II 


42/258 


43/258 


20/258 


105/774 




(16%) 


(17%) 


(8%) 


(14%) 


Level III 


31/315 


24/315 


21/315 


76/945 




(10%) 


(8%) 


(7%) 


(8%) 


Lovel IV 


17/417 


15/417 


40/417 


72/1251 




(4%) 


(4%) 


(10%) 


(6%) 


Level V 


30/385 


18/385 


52/385 


100/1155 




(8%) 


(5%) 


(14%) 


(9%) 


Level VI 


20/321 


15/321 


29/321 


64/963 




(6%) 


(5%) 


(9%) 


(7%) 



GE/age GE 



Age Total of 

all three 
variables 



•The nuntoer of rasponses on each posttest ranged from 39 to 
159, and averaged 83 responses per item. Significant 
correlations presented in this table were tallied in terms 
of an average significant coefficient of — .25 (at the 
.01 level of significance) for 83 cases (df * n - 2 * 81). 
Any correlation coefficient computed on items with 
difficulty level greater than .90 were not included with 
significant correlations due to instability of the index at 
extremes of item difficulty. 
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multiplication when presented with visual representations. 
However , it should be noted that the average age of those 
represented was somewhat high, about 8 years on the average. 
This result occurred because most second grade and many 
third grade students began in Level I for Multiplication and 
Division. Very few first grade children were represented, 
and use of all Level I tests should be validated on a sample 
of these students . 

Levels II through VI - At this point results become too 
numerous to record in narrative and are presented in 
tabular form (Appendix XI) for cases in which negative or 
otherwise exceptional results occur. 

Pretest Data 

The approach just described for analysis of posttest 
responses would not be appropriate for pretests. These 
are administered for the purpose of writing prescriptions 
and a wide range of performance is expected. Some students 
may get a substantial number of items right, thereby 
indicating the need for only a rather short prescription to 
clear up a relatively minor area of misunderstanding. In 
other cases, students may miss almost all the items related to 
a behavioral objective. Therefore, measures of pretest item 
difficulty or discrimination have little meaning. 
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If a Htudcnt attains tho mastery level on part of a 
pretest, he must later pass the corresponding part of the 
posttest. Should the pretest be too easy or too difficult 
with respect to the learning materials or the posttest, 
various difficulties might irise. Only one source of data was 
available to detect this problem. Teachers were requested 
to report difficulties related to testing on Incident Reports. 

Of the 300 or more Incident Reports received, about 
40 concerned circumstances involving pretests. Some of these 
reported errors or problems were due to the students' 
difficulty in interpreting the questions. All of these were 
corrected prior to reprinting of the tests for usage during 
the 1971-72 school year. However, a more frequently reported 
trouble was attaining mastery on the pretest and failing 
the posttest. Most of these cases were explainable in terms 
of posttest inadequacies just listed. Those not in this 
category were probably due to the fact that pretests usually 
contain more worked examples and hints to help students inter- 
pret notation. The fact that, in some cases, this help was 
insufficient to insure attaining posttest mastery after some 
time interval, is not reason for changing the pretests in the 
opinion of the test developers. 

All that this circumstance requires is a short 
prescription to permit the student to make up the deficiency. 
Based on Incident Reports received, it was not possible 
to identify the pretest areas definitely requiring revision 
due to being too easy. 
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S t. udcn t. _1 nyolvaneiv t i n _ J MS OjM'ia l i on 

The original formative evaluation plan (submitted lor 
funding by the Division of Educational Laboratories (of the 
U. S. Office of Education) called for systematic observation 
in classrooms to determine certain operating characteristics 
of IMS. It was thought that various procedures# such as 
prescription writing# checking work, and arranging for indivi- 
dual help might be improved upon as a result of these findings. 
However, part of the funding for the evaluation was withheld 
and the decision was made to delete the classroom observation 
phase. This decision was made because of the relatively large 
expense involved and the unavailability of trained personnel 
within the Laboratory for this work. Recruiting and training 
observers would have made this phase of the evaluation even 

more costly. 

To obtain some information in areas which might have been 
covered by observation# teachers were asked to respond to 
questionnaire items regarding the extent to which students 
could and did take responsibility for operation of the system. 
These questions appeared in Teacher Survey #2 (Appendix V b) 
after it was found that a related question on Teacher Survey #1 
(Appendix V a) was too general in its application. 

Results for question 8 of Teacher Survey #2 
(193 responders) arc as follows: 








% encouraging 
student 
involvement * 


„ of students 
successful ** 


Task 




Grades 

1-3 


Grades 

4-6 


Grades 

1-3 


Grades 

4-6 


Score 


Skill pages 


36% 


91% 


44% 


70% 


Score 


Check-up tests 


13% 


43% 


51% 


70% 


Score 


pretests 


9% 


21% 


36% 


78% 


Write 


prescript ions 


20% 


68% 


33% 


63% 


Obtain and return 
materials 


86% 


99% 


71% 


85% 



When asked what proportion of their students could score 
accurately their own check-up tests without regard to 
whether the practice was encouraged or not, responses 



were as follows (question 7) s 

Percent of students able 
to score check-ups 

0 - 25% 

25 - 50% 

50 - 75% 

75 - 100% 



No. of teachers responding** * 
Grades 1-3 Grades 4-6 

73 24 

7 21 

5 16 

6 25 



The general tone of these results may be characterized by the 
following observation. In grades 4 — 6, /2% of teachers 

estimate that 25% or more of their classes can score checkup 
tests accurately. Yet only 48% of these teachers encourage 
the practice. 



* Not counting small numbers of omissions. 

** Average of estimates by teachers encouraging student 
involvement in task. 



***There were 16 omissions. 
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Therefore , it is strongly recommended that teachers be 
encouraged to have students experiment and try to manage 
their own progress through IMS. As noted on page 63, in tne 
absence of any aides whatsoever, students at Countryside 
Elementary School were able to make progress through the system 
at somewhat greater rates than those of Urbania Elementary 
School where student involvement was less. 

Teacher Survey #3 also had questions regarding student 
involvement. See Appendix V c, items 13-17. The percentages 
reported there are the proportion of -"true" responses according 
to grade taught, grades 1 - 3 as compared with grade 4-6. 

As seen from these results, even in the lower grade a sub- 
stantial proportion of responsibility may be assigned to students, 
students. 



Aide Usage 

Of the 23 schools in the evaluation, fourteen used paid 
aides. They averaged about one full-time aide for every four 
or five teachers, but ran from a high of one full-time aide 
for each two teachers to a low of one aide for 13 teachers. 
Two schools with paid aides also used volunteer aides, but 
these were not the schools with the fewest aides per teacher. 
EC's estimates of time saved per teacher per day by paid 
aides ranged from 30 minutes to 150 minutes and is somewhat 
related to the number of paid aides per teachers. All but 
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one school reported that time saved per teacher by paid aides 
closely approximates or exceeded aide time available. 

Volunteer aides spent from 30 to 60 minutes per day with 
each teacher and reportedly saved those teachers from 20 to 
120 (!) minutes of work per day. These aides served in eight 
schools, two of which also had paid aides. 

Three schools used no aides at all. The rates of progress 
within IMS for students in these schools (one was Countryside) 
were as high as or higher than those reported by similar schools 
with aides. 

The above results regarding aide usage are summarized from 
a questionnaire sent to all EC’s (see Appendix VII) . 

Classes Not in IMS 

The EC questionnaire (Appendix VII) also sought information 
regarding the reasons for classes not entering IMS after 
being scheduled to do so originally. Of the 23 evaluation 
schools five reported classes not entering IMS due to materials 
shortages. In four of these schools, 13 classes have not 
entered IMS for this reason. Specific difficulties were 
the following: 

School layout requires additional materials (1 school) 

Manipulative materials needed (1 school) 

Not enough carts available (1 school) 
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Level VII and higher materials needed (2 schools)* 

Heavy concentration of students at lower 

levels requires more materials (2 schools) 

Accordingly , some schools had multiple problems, but 
app.irently these were not widespread. The fifth school was 
originally scheduled for 17 classes in IMS but reported 
only five at the time of the survey. In this case, serious 
multiple difficulties prevented full usage, but these were 
only partly due to materials shortages. 

In total, 264 classes were originally scheduled for 
IMS in the 23 evaluation schools. At the time of the 
survey, March 1, 1971, 207 classes were participating, and 
very few were added before June. In addition to materials 
problems, reasons for classes not participating are as 
follows: 

Decisions not to use IMS in first grade. 

School organization changes. 

Teacher preference. 

While all of the above reasons and seme of the materials 
problems (carts and manipulative materials availability) 
were beyond the control of the Laboratory, every effort was 
made to assure users of adequate availability of materials 
and assistance in other areas. 

♦A number of schools expressed the need for these materials, 
but only two actually kept classes out of IMS for this reason. 
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Chapter IV 

Results from Intensive Evaluation Schools 

The purpose of this chapter is to relate results of system 
operation as reported in Chapter III with standardized te 3 t 
scores as collected in the four intensive evaluation schools 
(see pages 17-18) . Data reported here cover only the 1070 
students at these schools who were present on both days of 
standardized testing. By grade level they are distributed 
as follows: 

Grade No. of Cases 

1 70 

2 153 

3 293 

4 342 

5 152 

6 60 
1070 

Census reports for the four intensive evaluation schools show 
a total of approximately 1500 students using IMS in these schools. 
The bulk of the deficiency must be accounted for in terms of 
absences due to illness and turnover in the sample. One third 
grade class of about 25 could not complete the second half of 
the ITBS. While attrition is rather high, in the opinion of 
the author it represents ..o readily classifiable source of bias 
insofar as results of this study are concerned. Specifically, 
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there w«is no exodus of white students ns a result of rc'cent 
change}; in racial coni|xv; i L i on of the schools. 



With respect to standardized test results, it is important 
to keep in mind the differences between materials covered and 
level of mastery expected in IMS as compared with the conventional 
curricula on which test standardization is based. For example, 
consider two low ability students, one in IMS and the other in 
a conventional classroom: it is possible that the two may not be 

taught any topics in common during the school yearl The 
conventional classroom student will be exposed to but will not 
master grade - level topics, while the IMS student will achieve 
mastery of topics which are below grade-level. Therefore, compar- 
ing the achievement of these two students with a standardized 
test covering mostly grade-level topics is meaningless. 



This rather obvious discrepancy led the developers of IMS 
to set no goals with respect to below-grade-level students 
as determined by standardized test results. The developers 
did set a goal of one year's growth in standardized test results 
for students at and above grade- level on these same tests. 
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In order to present results relating to this goal, it 
is necessary to draw a correspondence between the levels of 
IMS and those represented by grade equivalent (GE) scores from 
standardized tests. While the levels of IMS are not directly 
comparable to work at any given grade, there is a rough 
correspondence in terms of new topics introduced so that 1.5 
levels of IMS corresponds to approximately one year of a 
conventional curriculum. Figure 11 shows the grade level and 
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6th Grade 
Topics * 



5th Grade 
Topics * 



— IX 



— Vffl 



vn 



4th Grade 
Topics * 



3rd Grade 
Topics* 



— VI 



V 



— IV 



2nd Grade 
Topics* 



2 




GRADE 
LEVELS 1 



1st Grade 
Topics * 
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A 



IMS 

LEVELS 



Fig. 11 Comparison Between IMS-1 Levels and Elementary 
School Grade Levels. 



* Topic coverage as represented here is only approximate and varies 
according to curricula. 
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IMS level scales for purposes of comparison. Note that each 
scale has the number, one (Roman or Arabic), as its base or 
zero joint. Therefore numbers above six on the <| ratio- level 
scale represent work taught (luring the course of sixth ijrade. 
Similarly, numbers above T X on the IMS scale represent work 
within Level IX of IMS. 

After the initial administration of the standardized test, 
it immediately became apparent that growth in GE scores 
for above grade-level students would provide as little basis 
for evaluating IMS as is obviously the case for below-grade- 
level students. The reason for this difficulty is the low- 
ness of IMS placement achieved by above grade-level students. 
Thus almost all students in IMS, even those above grade- level 
on standardized tests, were placed in IMS so that the bulk of 
their work for the year was on below grade-level topics. 

The developers of IMS interpret this outcome as the consequence 
of the non-mastery approach in the conventional programs to 
which these students had previously been exposed. 



Grades Three Through Six 

Figure 12 illustrates the situation for 79 6 third through 
sixth grade students at four achievement levels: 

A - those whose ITBS GE score exceeds their grade by more than 



B 



. 5; 

those whose ITBS GE scores lie between their grade and .5 
above; 

those whose ITBS GE scores lie between their grade and .5 
below; 

those whose ITBS GE scores are more than. 5 below their grade. 
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Fig. 12 Initial and Final Mean Grade Equivalent Scores and Average IMS Level Scores 
for Four Achievement Groups - October-May, 1970-71, Grades 3-6. 
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The left bar for each group represents the October and May 
mean ITBS GE scores. The right bar represents average IMS level 
for the group at the time of entry into IMS and at the end of 
the school year. There is little difference from grade to 
grade in terms of group membership or achievement characteristics 
At the beginning of the year, the average grade assignment 
of students in Group A was 3.8, those in B 3.8, in C 3.9 
and in D 4.2. Therefore, Figure 12 strongly and uniformly 
across grade levels illustrates that grade placement scores 
are much more highly discrepant with respect to level of 
mastery for the higher achieving students. 

Consider Group A: the average student in this group 

was approximately a beginning fourth grader but began IMS only 
at about Level IV. This same average Group A student covered 
1.72 levels of IMS during the year, thus barely working his 
way into topics normally introduced in the fourth grade. Thus 
it is not surprising that his GE score on subsequent testing 
did not increase markedly. Much the same analysis follows 
for students in Groups B and C, in which many students never 
even worked their way up to grade level topics . 

Group D, however, deserves special attention. The average 
gain in GE scores here is substantial for such a group. While 
there is probably some gain due to regression and readministra— 
tion of the same test, such an outcome shows the extreme 
effectiveness of IMS in motivating low- achieve rs . Often a group 
of this type has all its score gains attributable to extraneous 
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effects, such as maturation and regression. But a GE score 
gain of .75 on the average in only seven months is far too 
substantial for such explanations. A further consideration 
with respect to Group D is the fact that their IMS placement 
did not represent nearly as much of a discrepancy with respect 
to initial GE scores as was the case for the other groups. 

(In viewing the other groups’ GE scores it should be remembered 
that the measurement period was seven months, not a full school 
year. ) 

Looking at Figure 12 again, it appeals that GE gain is 
much lower for those with greater discrepancy between IMS 
placement and initial GE score. At the same time it appears 
that larger IMS gains and higher ability may modify this out- 
come. (Note that Group A had higher GE gain than Groups B 
or C and aDso had higher IMS gain.) To investigate this 
conjecture, five predictor variables were entered into a 
stepwise multiple regression program to predict gains in 
GE scores for students in grades three through six. The 
five predictor variables are: 

1) Difference between initial GE score and IMS placement 
(scores expressed on a common scale) . 

2) Average IMS gain score. 

3) Ability defined as the quotient, initial GE score/age. 

4) Difference between grade and initial GE score. 

5) Difference between grade and intital IMS placement 
(both converted to a common scale) . 

Only the first three variables entered the prediction equation 
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still accounting for a statistically significant i*roi>ortion of 
the variance of the GE gain scores. Of those three determiners, 
1) is by far the most powerful, followed by 2). Ability (3) 
accounted for a small but statistically significant proportion 
of the variance of the GE gain scores. The actual regression 
equation is as follows: 

47X + . 30Y + .32Z - .35 = 3, 
in which 

X = difference between initial GE score and IMS 
placement measured in IMS level units . 

Y = gain in average IMS level over three fourths 
of school year. 

Z = initial GE x 100/age in months. 

G = gain in GE over seven months. 

This result confirms the conjecture that gain in GE 
scores is largely a function of how far back a student is placed 
in IMS compared to his position as measured by a standardized 
test. The further back he is placed the less he is likely to 
gain. Further, the more he gains in IMS, the more he should 
gain on the standardized test. Ability has something to do with 
this gain but plays a relatively minor role. (Its addition to 
the prediction equation raised the multiple R from .45 to .51.) 

Now, of course, the question that standardized test results 
v/ere expected to answer has changed. We know that being put 
back in IMS reduces gains in GE scores and that covering 
more ground in IMS increases them. What we need to estimate 
now is how well those who were put back will catch up. 
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hero did not work on IMS the fuJl school /oar. A fairly 
close estimate in that the average student used the IMS materials 
three- fourths of the school year. Thus, the IMS gain means 
should be multiplied by four-thirds to estimate average 
progress in a full school year. In Group A, the observed 
average IMS gain of l. 7 2 becomes 2.29 for a full school year. 

At the end of the school year, the average student in Group A 
was in grade 4.8, thus having 2.2 additional years of elementary 
school. Therefore, he should cover 2.2 x 2.29 or 5.04 more 
levels of IMS in elementary school. Adding 5.04 to 5.61, 
the point at which he finished the 1970-71 school year, yields 
an estimate of 10.65 for a final IMS level. This means that 
the average student in Group A (and all those above average 
in Group A) should more than finish nine levels of IMS. For 
Group B, the estimate of final IMS level is 8.48, in which 
case about 7% of students in this group might be expected 
to finish nine levels of IMS, assuming a reasonably normal 
distribution of final average IMS levels with a standard 
deviation of about 1.0 (see page 33). Virtually no students in 
Groups C or D may be expected to finish nine levels of IMS 
by the end of the sixth grade. 



Multiple regression yields another approach to predicting 

progress within IMS for students in grades three through six. 

i 

The following variables v/ere selected for predicting average 
IMS levels at the end of tie school year: 
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1) Difference between initial GE score and IMS placoment 
(both scores expressed on a common scale) . 

2) Ability defined as the quotient, initial GE ncore/age. 

3) Difference between grade and initial GE score 

4) Difference between grade and initial IMS placement 
(scores expressed on a common scale) . 

Only the first two of these predictors entered the prediction 
equation accounting for a statistically significant proportion 
of the variance of the IMS gain scores, and the second (ability) 
accounted for only a very small proportion of this variance 
(multiple R increased from .49 to .51 with inclusion of 2) 
as a predictor). The resultant regression equation is: 

. 19X + . 21 Z + .41 = I, 

in which 

X = difference between initial GE score and IMS 
placement measured in IMS level units. 

Z = initial GE x 100 /age in months 

I = gain in average IMS level over three-fourths of 
school year. 

Application of this equation to the existing sample 
groups A, B, C and D, would no more than duplicate the information 
of Figure 12, but this result further confirms the conjecture 
that discrepancy between mastery level end GE is the major 
factor in predicting progress within IMS when the system is 
first introduced in grades three through six. 
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First Grade 

Comolete data were available for only 70 first grade 
students. Of these, 58 had very high GE scores, averaging 
2.2 d. Like their older counterparts in Group A above, they 
had very low IMS initial placement, which averaged 1.69. 

Their average GE gain was .45. Other circumstances should 
be mentioned with respect to this group. Many of them entered 
IMS somewhat late in the year after the normal socialization 
processes has been completed in first grade, and, in some 
cases, use of the materials was sporadic at best. 

The developers of IMS were disappointed with this usage 
for two reasons. They had hoped that bright first grade students 
might be introduced to IMS much earlier in the school year and 
that nearly all average or below first grade students would 
enter the system at least by mid-year. This mode of usage 
was followed by some schools, who reported it very successful. 
However, in the four schools in which standardized test data 
j were collected, only more limited usage was reported. 

i 

| As a result of this outcome a special EC meeting was 

| devoted to collecting impressions regarding IMS usage in 

j the first grade. These results are reported in Chapter V. 

Second Grade 

In second grade, three schools, Enterprise, Academia and 
Urbania used IMS for the major portion of the school year in 
classes in which standardized test data had been collected. 

In these cases, experience paralleled almost exactly what 

o 
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was observed in the higher grades. Placement was much lower 
than GE scores. Progress in IMS was greater for those with 
initially higher GE scores, and GE score gains were modest 
for the seven month period between test administrations. 
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Inter-School Differences 

While the results presented above are probably somewhat 
representative of educational settings in general, it is of 
interest to review results from specific schools. The char- 
acteristics of the four intensive evaluation schools are 
quite distinct and deserve individual consideration. 

Figure 13 shows the mean gain in average IMS level for 
each of the four achievement groups described earlier (see 
page 52) and for the four intensive evaluation schools, 

Academia, Countryside, Urbania and Enterprise (see pages 17-18) , 
As was the case for the total sample, three schools show a 
substantial reduction in amount of IMS material covered as 
relative GE scores decrease. This phenomenon is most 
pronounced for Adademia but is essentially reversed for 
Countryside. Further, there are substantial differences in 
coverage of IMS for-students of the same relative achievement 

levels. 

Application of statistical tests of the significance of 
these differences would be inappropriate and would only 
belabor the point. Figure 13 shows that achievement within IMS 
was quite different from school to school, even for groups 
initially comparable in terms of standardized test scores. 

The causes of these differences cannot be established from 
available data, and it is possible only to offer conjectures 
as to their origins. 
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Fig. 13 Mean Gains in Average IMS Level by Schools and by Achievement 
Groups, October- June, 1970-71, Grades 3-6. 
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The students at Enterprise gave the appearance (to the 

author) of being tense and seemed easily agitated. Some teachers 
there expressed discouragement regarding the achievement of a 
substantial proportion of their students. They attributed 
this lack of success and behavioral problems as well to inade- 
quate home environments and social or racial tension in the 
community. Therefore, it would appear that the mean gain in 
average IMS level for Enterprise (Figure 13) may have been de- 
pressed by factors external to the school. At any rate, the 
author observed nothing which would suggest any inadequacy 
on the part of the Enterprise staff. That IMS did permit a 
noticeable degree of achievement at Enterprise is a tribute 
both to the school and to IMS. 

Countryside gave the impression of being a very calm school, 
IMS was introduced at each grade level only after careful 
preparation of teachers, parents and students. Students seemed 
eager to learn. The fact that the lowest achievement group 
at Countryside had such high IMS achievement may be an arti- 
fact. The Countryside sample was heavily loaded with fifth and 
sixth grade students, some of whom had to begin IMS at rather 
low levels. Good management of the system by teachers and stu- 
dents permitted many such students to catch up quickly in areas 
of deficiency. It may be noted at this point that Countryside 
had no aides? the other intensive evaluation schools did. In 
the absence of aides Countryside teachers encouraged students 
to take as much responsibility for system operation as they 
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believed feasible. This approach may have motivated the stu- 
dents strongly. In any case a high degree of achievement was 
demonstrated (see Figure 13) . 

Urbania's initial standardized test score means are much 
like those for Countryside. However, IMS progress was much less 
on the average, even with the help of aides. Yet Urbania s 
general atmosphere seemed quite cheerful and supportive of 
educational activities in general. The lower IMS achievement 
at Urbania may have been due to a more exacting application of 
IMS procedural "rules" than at either Academia or Countryside. 

The aides at Urbania were strongly organized in terms of checking 
and returning students' work, and formalities regarding passing 
posttests and writing prescriptions were carefully observed. 

It is possible that strict adherence to schedules and 
regulations served to depress motivation and, in turn, 
achievement in IMS. On the other hand, it is possible that 
a combination of variables, possibly masked by the excellent 
facilities of the school, led to a slight depression of achieve- 
ment and that performance at this particular school is nearer 
the average than any of the others. 

Though not shown above, achievement within IMS for second 
grade students at Urbania was somewhat depressed also. The 
average second grade student at Urbania covered only .42 level 
of llCMS in approximately six months. In comparison, at 
Enterprise the average second grade student covered .65 level. 

At Academia, the average second grade student covered 1.02 
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levels of IMS. Countryside completed the introduction of IMS 
in only grade three through six during 1970-71. 

Figure 14 shows mean GE gain scores by school and ability 
groups in the same format as Figure 13. These results are not 
as dramatic as those of Figure 13 but are offered because the 
reader may wonder the extent to which accelerated IMS coverage 
was accompanied by higher standardized test scores. Countryside 
record appears best in this respect. This result parallels 
the finding that IMS gain is a significant predictor of grade 
point gain in a multiple regression aquation. 

The surprisingly low gains i or Groups B anc C at 
Academia are not readily explainei in view of generally high 
rates of IMS achievement. The number of cases involved is 
relatively small and did not adversely affect establishment 
of regression equations reported earlier. This outcome may 
be due to inappropriateness of the older, dated ITBS form 
used with respect to Academia's somewhat advanced curriculum 
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Evaluation by Consultant s 
Structure and content are the two aspects of IMS regard- 
ing which the Laboratory sought evaluation by acknowledged 
experts. Dr. Robert M. Gagne of the Florida State university 
was engaged to review IMS materials and procedures from the 
standpoint of the learning theories with which IMS is com- 
patible. Appendix VIII is Dr. Gagne's report. 

With respect to this report, the reactions of the IMS 
staff should be reported here insofar as they relate to changes 
which may be made as a result thereof. All agreed that a 
number of the behavioral objectives should be restated to 
sharpen the precision with which they specify criterion 
behaviors. Also, it was agreed that empirical studies on 
interdependency of skills might well make possible new 
approaches to prescribing learning which would be much 
more efficient. However, it was felt that such an undertaking 
was not feasible with currently available levels of funding. 
Similarly, redesigning the pretesting phase of IMS to provide 
more precise diagnostic information was believed desirable but 
also very costly. This development subsumes a more precise 
characterization of teaching pages and activities to permit 
maximum usage of more information from pretesting. Dr. Gagne's 
recommendation for developing and characterizing activities 
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of the system in terms of application, transfer of learning 
and problem solving is certainly feasible if given modest 
support and good direction. 

Testing is one phase of IMS which it was felt may have 
been misinterpreted by Dr. Gagne. If IMS is properly 
managed, students should feel no pressure from testing. 
Placement tests are to be used only when a student enters 
IMS, and pretests are used only for prescribing work in 
skill folders. If all has gone well, the posttest should 
not be a challenge but, instead, an opportunity for the 
student to display what he has learned and at the same 
time obtain more practice. 

Dr, Joseph Scandura of the University of Pennsylvania 
reviewed IMS from the standpoint of mathematical content 
and suitability for fostering attainment of the behavioral 
objectives. He made a substantial number of specific 
suggestions regarding format, notation, terminology and 
the extent to which materials conformed to their related 
objectives. These comments, along with his suggestions 
for further development of IMS comprise Appendices IX a and b 
Dr. Scandura' s specific recommendations for revisions 
were presented to the IMS staff for their consideration and 
are discussed further in Chapter VI (page 88). 

Another area of Dr. Scandura' s report consists of pro- 
posals for enhancement of the IMS program in three related 
areas t arithmetic skills, critical reading, and logical 
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thinking. The importance of training in these areas and the 
benefit to students using IMS cannot be denied. Not only 
should development of critical reading and logical thinking 
skills help students in IMS, the benefits should extend 
to other curriculum areas. While arithmetic skills are 
taught in IMS, development of additional activities in 
this area is certainly appropriate. 

It is strongly recommended that the proposed development 
work outlined by Dr. Scandura be given highest priority 
when program resources become available for such activity. 

Teacher Characteristics 

Teacher Survey #1 (Appendix V) covered the 175 teachers 
using IMS in December. These, included four first grade teach- 
ers, 33 second, 35 third, 40 fourth, 35 fifth and 24 sixth. 
Four respondents did not indicate grade taught. Only 6% 
of respondents had used IMS the preceding year, but 13% 
had had experience with other individualized mathematics 
systems. Average experience of the 175 teachers was 10.8 
years, and 17% held masters degrees. Over 95% felt their 
background in mathematics adequate for teaching with IMS. 

Later surveys covered approximately 200 teachers, as more 
classrooms began using IMS, but the distribution across 
grades remained approximately the same. 

Questionnaire Results 

Three questionnaires were administered in December, 

March and May (see Appendix V) . The first two requested 
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responders ' names and the third was anonymous. On the first 
two, responders were encouraged to omit items if in doubt 
but on the third were requested to respond to all items 
(see cover letters jand instructions in Appendix X) • 

As may be seerj from the questionnaires themselves, a 
wide range of topics was covered. Many of these are discussed 
elsewhere in appropriate sections of this report. For example, 
responses regarding materials revision are covered in Chapter 
VI, and those regarding effectiveness of summer workshops 
are covered in this chapter (page 76 ) • Responses concerning 
student involvement, with system operation are discussed 
in Chapter III. What remains to be summarized here are for 
the most part questions of opinion or judgement on the part 
of responders. Responses to a few procedural questions 
were simply tabulated and reported to the IMS staff (e.g., 

"Should keys to skill pages be laminated?") . 

Teacher Survey #1 

This survey showed almost unanimous agreement that the 
materials are attractive and generally effective with students 
(see Appendix Va for responses to IMS Goals, items 1-5,9,10,15). 

The questions regarding activities, materials supply, 
students taking responsibility for operation of the system, 
and parent opinion and understanding of the system suggested 
the need for more detailed questions on the second questionnaire. 

Question 17 concerned overall effectiveness of IMS 
compared with the system used previously. About 42% of 
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responders chose not to answer the question. Of those 
who answered, 69% chose IMS as the more effective system. 

A large proportion of responders cited the need for some 
kind of test to precede entrance into Level I. The IMS 
staff responded with publication of the IMS Initial Screeni ng; 
Device . Its introduction was reported extremely helpful 
by all users (see IMS Problems, items 1-2) . 

Very few responders claimed that older students reacted 
badly to lower level materials (IMS Problems, item 13) . 

When asked if some mathematics system other than IMS 
might be more suitable for some students in their classes, 

47% of responders replied positively. Almost all of these 
went on to specify low achievers as not profiting from IMS 
compared with a teacher-centered, drill-oriented curriculum. 
This outcome is in direct conflidt with the facts as reviewed 
on pages 54-55 regarding Group D, those six or more months 
below grade level according to standardized test results. 



Teacher Survey #2 

There were 193 responders to this survey administered 

in March. The increase over Teacher Survey #1 was due to 

additional teachers using IMS. Grade taught were as follows: 

2nd 3rd 4th 5th 6 th Tota^ 

44 40 40 34 26 193 



No. of respondents 



1 st 

9 
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Question 5 pursued the problem of availability of 
materials. Responses were as follows: 



Shortage of 

Special Pens or Pencils 
Skill Folders 
Manipulative Materials 



Percent Responding 
Now Previously 



27% 
2 3% 

24% 



22 % 
13% 
15% 



Apparently shortages of one kind or another affected many 
classrooms using IMS. Apart from the simple need for 
larger quantities there is apparently some need for better 
usage arrangements within schools , since responses to 
question 6 were as follows: 



Lack of carts or storage space has caused 
supply problems 

School layout has prevented efficient use of 
materials 



Percent Responding 
14% 

14% 



These responses and related data were reported to the IMS 
staff. Along with other results, these outcomes led to the 
establishment of a stocking model responsive to school layout 
and achievement levels of students. This model was used 
for stocking schools for the 1971-72 school year. 

Question 9 regarding prevailing student opinion of IMS 
had the following responses: 



162 Favorable 



1 Unfavorable 



27 Mixed 



3 Omitted item 



Question 11 had responses as follows regarding parent 
understanding of certain features of IMS: 
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Percent of Teacher Responses 



Almost all 

Parent Understanding-* u n ^ ers t-and 
IMS Feature — 



Each student works at his own level 
Each student works at his own speed 
Completion of a level represents 
mastery/ not exposure as in th~ 
textbook approach. 



61% 

56% 

25% 



Many Do not Ttoin 

Understand Omitted 

25% 14% 

29% 15% 



54 % 21 % 



The large proportion indicating parental non-understanding 
or omitting the question suggested a substantial problem 



area for IMS. It was the case that many schools did not make 



use of available resources in explaining IMS to parents, 
while others showed an IMS film at PTA meetings or had IMS 



representatives speak to parent groups. Still others 
scheduled demonstrations of IMS or used other local resources 



to present this facet of their school programs. 



The apparent lack of parent knowledge regarding the 
"mastery" aspect of IMS was especially disturbing . It was 
pointed out to the EC's that one way or another many parents 
would come to know that by reaching, for example, only 
Level VI or VII of IMS in elementary school, their children 
would not have been exposed to certain topics typically 
in sixth grade textbooks. Only if they understand that such 
exposure is no substitute for really being able to perform 
in lower areas will they appreciate IMS fully. 



Of the responses to question 10 regarding prevailing 
parent opinion of IMS, responses were as follows: 

70 Favorable 10 Unfavorable 

74 Mixed 29 Omitted item 
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An interesting subset of these data is the responses of 
those teachers who indicated that they thought almost all 
parents understood at least two of the three features of 
IMS discussed earlier in this section. Of the 193 teachers 
responding 105 are in this category. Their responses as 

to prevailing parent opinion were as follows: 

5f> Favorable* 5 Unfavorable 

4 0 Mixed 4 Omitted item 

The proportion favorable in this group is 53% as compared 
with 16% for the remaining group when these are removed from 
the total. In other words, 56 of the 70 favorable responses 
regarding parent opinion came from teachers who also 
thought that almost all parents understood at least two main 
features of IMS. Of course, teacher judgement of parent 
opinion is only a substitute for the actual expressions of 
opinion but this result is certainly strong evidence in 
favor of making every effort to keep parents well informed. 
Teacher training procedures were modified as a result of 
these findings to emphasize the need for iftforming parents 
about IMS . 



Four questions concerning teacher opinion of IMS had re- 



sponses as follows: 



% of Teacher Responses 
Yes No Omit 



Do you find IMS, and materials gener- 
ally effective for helping students 
achieve IMS objectives?* 



5% 13% 



★ In answering this question, teachers were asked to delete from 
consideration seven skill folders undergoing substantial revisions. 
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% 

Yes 


of Teacher 
No 


Responses 

Omit 


Does provision of activity pages 
in addition to work pages assist 
in achievement of IMS Behavioral 
Objectives? 


61% 


11% 


28% 


Does IMS enable the te ache r to 
spend more time giving individual 
assistance to students? 


69% 


23% 


8% 


Does IMS seem more effective than the 
mathematics system you used 
previously? 


70% 


13% 


17% 



These statistics speak for themselves, but two points are 
noteworthy. On the negative side, the proportion of omissions 
to the activities question was disappointing. Although 
activity pages through Level IV had been distributed some 
time prior to the survey, apparently many teachers had not 
used them extensively enough to answer this question. On 
the positive side the proportion reacting positively to the 
last question is much higher than to a similar question on 
the first survey, when 42% of responders omitted a similar 

question. 

Teacher Survey #3 

Essentially the same teachers responded to this survey 
as to the preceding one. Since Survey #3 was anonymous and 
responders were requested to answer all questions, omissions 
were few, never exceeding 5% of responders. Appendix V o 
shows actual proportion of responses. 

Almost all responses to this questionnaire were highly 
positive. One exception was the continuing number of 
teachers claiming not to use IMS activity pages (item 6). 
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Because of this finding teacher training procedures were 
revised to give greater emphasis to activities. 

Item 19 shows that a large majority of teachers 
believed IMS more effective than the textbook approach 
across a wide variety of situations. When taking all aspects 
of effectiveness into consideration (item 20) over 91% 
preferred IMS over the textbook approach. 



Evaluation of Workshops 

Three workshops for teachers and one for principals were 
held during the summer of 1970. At the conclusion of each 
workshop, the anonymous questionnaire, IMS Workshop Evaluation 
Form, was administered. A total of 131 responders filled 
out this instrument. 



Of the 39 items, the first 27 lent themselves to an 
ordinal scale, 1 for the least positive response through 
5 for the most positive. The average response over these 
27 items and the 131 responders was 4.15. Accordingly, 
almost all participants chose very complimentary responses 
to these items. There was little difference from one workshop 
to another. The latter items, while not readily scalable, 
were also answered in a highly complimentary fashion. 

For example, only one responder - said he would not recommend 
the workshop to others (item 38) . The open ended questions 
were answered by most responders with no negative remarks. 
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While these results were reassuring, the matter of 



teacher training was pursued 



further through Teacher Surveys. 



Teacher Survey #1 asked if the respondent had attended an 
IMS workshop but failed to clarify whether it was one of the 
"official” workshops given by the IMS staff in the summer 
of 1970. Questions 2, 3, and 4 of the Teachers Survey #2 
were included to clarify this situation with results as 



follows: 

Quest. #2. . 3-Type of Training 

IMS Summer 1 70 Training 
Other training for IMS 
No Formal Training 





Question 


#4 


Prepara t ion 


Adequate 


Yes' 


No 


Omit 


45 


18 


5 


38 


20 


4 


20 


30 


13 



As seen from the table the porportion of teachers who felt 
unprepared was much higher among those with no formal 
training. Discounting omissions, 60% of those with ho 
formal training felt inadequately prepared, versus only 29% 
of those with IMS training and 34% of those with other 
formal training or experience. However, the fact that 29% 
of those with IMS training felt inadequately prepared was 
considered an indication that training procedures could be 



improved . 

To investigate this matter further, each of the 23 
teachers who answered negatively or omitted the question 
on adequacy of training by IMS was contacted by letter. A copy 
of the original IMS Workshop Evaluation Form was enclosed, 
and the respondent was asked to fill it out again in light 
of the year's experience. 
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Thirteen teachers responded, and the replies were 
highly consistent, though from a number of different schools. 

All 13 claimed that the main weakness of the workshops was 
insufficient attention to the IMS materials themselves. 

They found themselves unable to recognize and use the materials 
effectively when confronted with the actual classroom 
situation. Most recommended having the teachers go through 
the same steps as the students during the workshop or establish- 
ing demonstration classes with which workshop participants 
could interact. This outcome was reported to the IMS staff, 
and workshop plans for the summer of 1971 were changed to 
include more opportunities for participants to work directly 
with IMS materials. 

Cost Data 

Records were maintained by the IMS staff covering all 
aspects of production, teacher training and monitoring. 

Two outcomes may be noted as a result of analyzing these data. 

A proposal was submitted to the National Science Foundation 
for training of 500 teachers to use IMS during the 1971-72 
school year. The budget for this proposal, which was 
subsequently funded, was based on cost data for earlier 
workshops. Monitoring services were included in this budget. 

The cost of the materials themselves was established at 
approximately $10 per student for the first year's stocking. 
Revisions and replacements during the subsequent four years 
were estimated to cost $1.50 per student per year. Thus 
the total five-year cost was estimated to be $16 or $3.20 
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per student per year. This cost might be expected to vary 
according to circumstances - the need to stock building:; 
or floors of the some school separately, or large quantities 
of lost or damaged materials. 

The estimated cost of $3.20 per student per year did 
not cover items provided locally, such as carts, supplies 
and manipulative materials (counters, games, scales, measuring 

devices, etc.). 

Monitoring Activities 

The IMS staff made about 30 monitoring visits during the 
course of the year. Many of these visits were for the 
purpose of dealing with specific problems, such as training 
new teachers, alleviating supply difficulties, etc. However, 
for all visits, detailed records of what transpired and of 
the monitor's impression were maintained. 

Without question, these visits served to make IMS more 
effective during the course of the year. For subsequent years, 
two observations came through quite clearly in the various 
monitor reports: 

1) Teacher training is \rital to the success of 'the 

program. Almost every teacher problem encountered 
was the result of lack of training. (It should be 
noted that about one- third of the teachers using 
IMS in 1970-71 had no formal training.) 

Strong administrative support of the program is 
a big factor in its success. Teachers can carry 
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a largo share of the burden, but supply shortages 
and scheduling inconsistencies can scuttle their 
best efforts. Also, principals may have a large 
effect on parents' acceptance of the program. 

Manipulative Materials Usage 

The greater part of an EC meeting was devoted to dis- 
cussing manipulative materials usage. Before the. meeting EC's 
had collected data on this phase of IMS and filled out the 
Supplies and Materials Questionnaire (see Appendix II) . 

This questionnaire was based on the Suggested Materials List 
for IMS (Appendix I), which had been provided to schools. 

The purpose of the meeting and data collection was to 

determine the need for revision of this list. 

The results are somewhat inconclusive. Every item 
in the "supplies" category was reported in use in at least 
some schools, though some much more frequently than others. 
Apparently supplies usage is highly idiosyncratic from school 
to school. What effect this may have on progress within 
IMS remains undetermined. 

in the "purchased materials" category there was similar 
lack of uniformity. A number of EC's expressed the need for 
items they had requested but never obtained. Pan balance 
scales was the most frequently mentioned item in this respect 
Clocks with articulated hands were also mentioned as needed 
but unavailable. Only eight schools reported the presence 
of an abacus, but six of these reported it not well used. 
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One of the two remaining starred this item as the s.nyle 
most valuable manipulative. All schools agreed that 
centimeter rods were absolutely essential to IMS. 

A number of suggestions were received for home made 
manipulative materials. These were turned over to IMS 
personnel for transmission to new schools. 
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First Grade Usage 

Almost an entire EC meeting was devoted to the subject 
of IMS in the first grade. Some of the EC's brought first 
grade teachers experienced in IMS at their schools. Out of 
this discussion came several suggestions for the use of 
IMS in first grade. 

All present teachers agreed that a school has to plan 
for aides , volunteer or paid, if IMS is to be successful 
in first grade. It is a necessity to have at least one aide 
in each room during the math period. More aides are 
desirable in order to have a smoothly running program. 

Morehead Elementary School, in Durham, North Carolina, 
had five aides in the first grade classroom when it started 
on IMS . By the end of the year they had reduced the 
number to three. If it is impossible to get the aides 
needed, some teachers suggested having the good readers help 
slow readers or getting fifth and sixth grade students to come 
in during the math period to help. 

EC's and first grade teachers Also agreed that the 
math period should be longer than 30 minutes. Otherwise, 
by the time the students are organized and have their materials, 
there is little time left for actual work. The suggestion 
was made t:o have three 50 minute periods a week rather than 
five 30 minute periods. 

Screening must be done, and the brightest students 
should be started in IMS first. At Deep Creek Elementary 
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in Chesapeake, Virginia, the students were to be screened at 
the end of the kindergarten year, so that those ready could 
start IMS at the beginning of first grade. 

A preparation program should precede actual work in IMS. 

It is necessary to get the students acquainted with the 
materials , vocabulary, and the operation of IMS. The 
students should be allowed to look at and handle all the 
materials. They should be instructed in the sequence of 
activities in IMS. The knowledge of the vocabulary of IMS 
is essential to the system's success. Many teachers 
suggested that the parents who are able should help the 
students at home with the new vocabulary. This would take 
some of the burden from the teacher and also would involve 
the parents and acquaint them with the system. 

A total of fifteen first grade classes used IMS 
materials during the course of the year. However, a number 
began somewhat late and in several only a minority of students 
participated. At Urbania approximately 30 first grade students 
participated. Their initial placement resulted in a mean 
average IMS level of 1.61. By the year's end, this mean had 
risen to 1.96 for a rather small gain of .35. 

At Academia Elementary, 46 first grade student! began at 
about the same point in IMS (mean of 1.64) but covered, on the 

1.24 levels. of IMS. Both groups were considered somewhat 
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above average in ability and both spent about the same 
amount of time with IMS during the year. This set of 
results, while not extensive, does illustrate that substantial 
differences in progress can arise, apparently due to the 
way the system is managed. 
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Chapter VI 

Revision of Materials 

Skill Folder Revision 
Revision data arise from several sources: 

Incident Reports 
Reports by Consultants 
Questionnaire Responses 
Posttest Outcomes 

To some extent all of the above have been reported 
earlier. It is the purpose of this chapter to lend some 
unity to these diverse results. In addition, the progress 
made toward revision will be cataloged, and recommenda- 
tions for further revision will be made. 

Mid-year Revision 

Teacher Survey #1 asked responders to list the skill 
pages or tests they felt needed immediate revision. The 
following skill folders had pages mentioned substantially 

more often them any others: 

Multiplication I, Skill 2 
Multiplication II, Skill 6 
Measurement II, Skill 1 
Division II, Skills 2, 3 
Time III, Skills 2, 3 

Based on teachers' descriptions of the difficulties 
encountered, revised versions of these skill folders 
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were written and tried with students at Morehead Elementary 
School in Durham. Further revisions were made following 
this testing. 

Though resources were limited, 12 revised sheets 
(24 pdges) were laminated and sent to schools. These 
pages were in Measurement II and Division II and, it is 
believed, cleared up the worst difficulties in these 
folders. The difficulties in Multiplication I and II 
and in Time III were more pervasive. It was recommended 
to all schools that students be helped through these 
skills without the usual requirements for posttesting. 

End of Year Revision 

In addition to revisions already made in Multiplication I 
and II and Time III but not distributed, a large number 
of changes were recommended through Incident Reports. 

Many of these were simply corrections of errors due to 
oversight. By way of more than 300 incident reports and 
the inspection of all pages not already revised, approxi- 
mately 250 pages were corrected or revised. 

In addition to revision just described, all folders 
of Fractions II and VI were modified insofar as termin- 
ology and format were concerned. These changes were made 
by the writing staff for IMS-2, the CIIS junior high school 
program. This group was also completing Levels VII - IX of 
IMS and made the changes in Fractions V and VI to accomodate 
the approach they had adopted for presenting Fractions VII - IX. 
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Revision described thus far completed the process insofar 
as preparation of materials for the 1971-72 school year is 
concerned. Schedules were established to arrange for replace- 
ment of part of these revised pages in the 1970-71 schools. 

In some cases, however, the revision was too trivial to warrant 
the replacement. In addition to the five folders listed earlier, 
approximately 80 sheets (160 pages) were recommended for 
replacement in the present schools. 



Revision to be Completed 

Revision work yet to be complete falls into two categories. 
The first covers the materials of Levels VII - IX. As 
mentioned earlier, these materials were not completed in time 
for use by students during the 1970-71 school year. Further, 
the developing and editing of these materials was done by the 
IMS-2 junior high school writing staff and represents a 
substantial departure in style and approach from the earlier 
materials. Story lines have been introduced in many folders 
so that prescription of isolated pages is no longer possible. 
(The student must go through the folder from front to back to 
preserve continuity and understand the context.) Also, sub- 
stantially more reading material has been introduced (sometimes 
unavoidably) . The new materials are more "discovery" oriented 
and less didactic. 

In view of the foregoing, no assumptions should be made 
regarding the materials of Levels VII - IX based on experience 
from the first six levels. A full scale review and revision 
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process is recommended based on the model established for 
evaluating the first six levels. 

The second incomplete area with respect to revision 
is the parts of Levels I - VI not covered above. Analysis 
of posttests came too late in the school year to permit 
acting on many changes indicated thereby. Chapter 111 
all posttest items which indicate a need for revision. In 
many of these cases, it is the test itself which should be 
revised rather than the related materials. Most materials 
problems covered by test analysis are the results of insuffi 
cient provision of practice for certain kinds of problems. 

The critique of the materials by Dr. Joseph Scandura 
(Appendix IXa) also calls for a number of revisions in 
Levels i - VI. Many of these suggestions overlap changes 
already made and changes indicated by posttest analyses. 
Further, whether some changes should be made is a matter o 
choice to be resolved by the IMS staff. Other changes listed 
in Appendix IXa are hardly debatable. 
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Chapter VII 

Summary and Recommendations 
Attainment of Goals 

The IMS Formative Evaluation Plan (on which this study 
was based) listed a number of goals. The plan also specified 
the bases for facilitating and measuring the attainment of 
these goals . The extent to which the goals were attained has 
been presented in earlier chapters; it is the purpose here to 
identify these goals more specifically and to summarize and 
interpret the results presented earlier. 

It must be emphasized with respect to all that follows 
that only the first six levels of IMS are covered. Only very 
limited conclusions regarding the latter three levels may be 
inferred from results presented here. 

Goal 1: The IMS Behavioral Objectives and materials are 

satisfactory from the standpoint of mathematical 
correctness and consistency and preparation for 
further study in mathematics. 

The main basis for determining the extent to which this 
goal was attained is the report by Scandura (Appendix IX a) . 
Scandura reports a small number of minor inconsistencies 
and errors but views these as superficial rather than basic 
faults. No entire skill folders or lines of development 
linking folders were found deficient. A number of those errors 
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he cited were corrected in the normal course of revision, but 
others remain to be corrected. Correction of a small number 
of deficiencies cited by Scandura depends on editorial rather 
than mathematical considerations. 

Goal 2: IMS Behavioral Objectives and materials are satis- 

factory from the standpoints of learning theory 
and the study of child development. 

The report by Gagne (Appendix VIII) concerns itself 
primarily with this goal. In general his conclusion is strongly 
affirmative to the effect that the goal has been met. The 
discussion in Gagne's report regarding the desirability of a 
more hierarchical sequence, of skills (depending on what the 
individual student has previously mastered) is not intended 
to imply a deficiency in the present system. Rather it is a 
recommendation for further development should funds for so 
substantial a project become available. 

Gagnfl suggests that testing may play too prominent a 
part in the present system. That it could, if misapplied, is 
readily acknowledged by the developers of IMS. However, 
many features of the teacher training program are designed to 
promote constructive use of testing. This topic is discussed 
in more detail on page 68 . 

Goal 3: The learning, materials and teaching aids are 

attractive. 

That this goal was met can be stated without qualification. 
Moreover, similarity of style and format probably means that 



the conclusion will hold for the latter three levels as well 
as the first six. Out of 196 teachers responding to a ques- 
tionnaire item regarding this goal, 195 answered positively. 

On a less systematic note, the IMS artists received dozens 
of spontaneous letters of appreciation from students using 
the materials. 

Goal 4: Providing activities and work pages assists the 

pupil in achieving curriculum goals more than 
work pages alone . 

In an early research proposal, it was planned that actual 
use of activities by students be recorded to permit comparison 
of achievement with that of students for whom no activities 
were prescribed. The funding agency did not approve that 
proposal, holding the procedures too extensive and costly. 
Therefore, in the presently reported research, it was necessary 
to rely on teacher opinion regarding the efficacy of activities. 
The situation is further complicated by the fact that activity 
pages for Levels III and IV were not distributed until mid- 
year; those for Levels IV and V were not completed in time for 
use by students. Further, 52% of teachers claimed not to have 
used available activities (Teacher Survey #3) , no doubt for a 
variety of reasons, some valid, some invalid. This result 
is viewed largely a failure of teacher training or the result 
of no teacher training in some cases, and yields recommenda- 
tions regarding further teacher training (see pages 75-76) . 
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At the end of the year, 92% of the teachers who had used 
activities agreed that they enhanced achievement over that 
based on skill pages alone. 

Goal 5 s IMS learning materials and teaching aids are 
pedagogically sound. 

Throughout the year, teachers forwarded reports of 
unsatisfactory student responses or reactions to materials. 
These reports were tabulated and summarized to locate problem 
areas. In addition, teachers were asked on two occasions to 
list pages or skill folders they thought should be revised. 
The areas reported by substantial numbers of teachers were not 
extensive and resulted in. a moderate number of revised pages 
(see Chapter VI) . 



In answer to several direct questions on Teacher Survey #3 
(Appendix V c) regarding the pedagogical soundness of the 
materials, 71% to 99% of teachers responded positively. 



Goal 6: IMS teacher training materials are effective. 

Goal 7 s IMS teacher training procedures can be accomplished 

j in an! average of 15 hours of instruction. 



The broader issue with respect to these goals is the 
overall effectiveness of the IMS teacher training program. 
As reported earlier (page 77), approximately one fourth 



of those trained found the training inadequate, citing 
lack of opportunity to become familiar with the IMS instruc- 
tional materials as the primary fault. To alleviate this 
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problem, new teacher training sessions were redesigned to allow 
more contact with materials. Both the original and revised 
schedules for these sessions cover about 15 hours of 
presentations on three days. The success of the change remains 
to be seen. 

With respect to Goal 6, a specific item explored this 
question on Teacher Survey #3 (Appendix V c) . Of those 
trained by IMS, 86% responded positively. In interpreting this 
outcome, IMS personnel believed it may have been influenced 
by the dissatisfaction referred to above rather than 
perceived inadequacy of the training materials themselves. 

Also, the question may have been misinterpreted as referring to 
availability of instructional materials. Apart from the 
preceding, there was other evidence of inadequate teacher 
training. However, it is impossible to separate these instances 
according to whether the teachers involved were among the 
35% trained by IMS. Reports by monitors, EC's, principals 
and others concerned; 

Failure to let students take responsibility for 
their own learning . 

Misapplication of testing procedures. 

Failure to prescribe activities. 

Failure to vary prescriptions according to pre- 
test results and student characteristics. 

In most cases these deficiencies were corrected during the 

course of the year. Nevertheless, regardless of their source, 

they strongly imply the need for a thorough and careful 

teacher training program. 
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Goal 8: The stocking procedures recommended by IMS provide 

adequate supplies of materials to meet pupil needs. 

Teacher Survey #2 (Appendix V b) revealed that 
approximately 25% of teachers continued to experience supply 
difficulties as late as March of the school year. All such 
problems were investigated. The results of these investiga- 
tions and study of levels and rates cf pupil progress within 
the system yielded revised stocking strategies for subsequent 
installations. 

Goal 9: IMS tests have good psychometric properties. 

Chapter II discusses in detail the results of 
investigation with respect to this goal. In addition , the 
tests were examined by experts in the field of mathematics 
and learning theory. In summary, it may be said that, except for 
isolated inadequate items, the tests are of very high quality. 

This conclusion i.fe further supported by the almost complete ab- 
sence of Incident Reports or other evidence of general difficulties 
concerning testing. A number of Incident Reports concerned 
isolated errors (later corrected) , but generalized problems 
were not found. 



Goal 10s Cost of IMS materials is $10 per pupil or less for 
schools of moderate size or larger . 

As presented in Chapter V, this goal was met with respect 
to schools entering IMS in the fall of 1971. Cost of IMS 
materials over a five-year period was estimated at $3.20 
per student per year. 
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Goal 11: IMS materials arc sufficiently durable to be reusable. 



As related to cost estimates, the projected period of 
reusability is five years with replacement (due to wear) 
of about 2% per year. No evidence from using schools indicated 
a likelihood of exceeding this allowance. Further, on Teacher 
Survey #3 (Appendix V c, item 9) only 8% of teachers expressed 
negative reactions with respect to durability. 

Goal 12: Students take responsibility for operation of the 

system. (50% of fourth grade students write their 
own prescriptions.) 



Statistics from Teacher Survey #2 (Appendix V b) give 
strong evidence that this goal was attained. These results 
are reviewed in Chapter III, pages 44-46. Reviewed in' Chapter IV 
is perhaps the more important finding that success in mathe- 
matics achievement seems positively related to the degree of 
student participation in operation of the system (see 
Inter-School Differences, page 61 ). However, it was the 
concensus that aides are required to introduce IMS in the first 
grade, and the preceding comment does not cover IMS usage at 
this level. 



Goal 13: Students achieve the Behavioral Objectives at the 

rate of one IMS level per school year on the average. 

Since the average student using IMS in 1970-71 covered 
1.28 levels of IMS, there is every reason to believe this goal 
was attained. Especially since this goal was made in less than 
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a full year.' Further, the apparently good psychometric 
properties of the tests used to certify progress within the 
system lends confidence to this conclusion. With greater 
familiarity with the system on the part of teachers and stu- 
dents, progress rates may increase. 

Goal 14: Students at and above grade level on standardized 

normative tests at the beginning of the school 
year gain an average of a one year grade equivalent 
score by year ’ s end under IMS . 

This goal was not met during the first year of operation 
for students at and above grade level. Instead, students 
six months below grade levels or lower had score gains equal 
to approximately a one year grade equivalent score change. 

This surprising outcome is attributed to the fact that brighter 
students were, cn the average, assigned to work far below 
grade level based on the mastery levels they could demonstrate 
at the beginning of the school year. As a result of this 
discrepancy, they spent all or the bulk of the year mastering 
bfblow grade level topics formerly learned more superficially. 

This apparently superficial learning did not hamper performance 
on the standardized tests but resulted in low IMS placement and 
prevented their reaching new topics during the year . The 
discrepancy for below grade level students as measured on 
standardized tests was not nearly so great, and these students ' 
grade equivalent change scores were consequently much higher. 

This entire phenomenon is discussed in great detail in Chapter IV 
pages 50 -.60 and represents a major outcome of the research 
reported herein. 
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With respect to the future# there is every reason to 
believe that brighter students will regain their positions with 
respect to grade level topics after a one to two year "catching 
up" period. Their progress rates within IMS were much greater 
than those of below grade level students. Therefore# the 
apparent deficiency in achievement should vanish for brighter 
students. Use of IMS beginning in the first grade should 
prevent this problem from developing. 







Recommendations for Further Research 

The primary questions remaining to answer concern 
Levels VII# VIII, and IX. As these levels come into use 
during 1971-72# the same goal areas just presented need to be 
investigated with respect to the new materials. This extended 
verification of system characteristics is important because the 
new Levels VII, VIII and IX differ in certain respects from 
the earlier levels. For a brief discussion of these differ- 
ences# see page 87 . 



In addition to monitoring the use of the new materials# 
it is recommended that data continue to be collected with 
respect to Levels I - VI , though perhaps on a smaller scale 
than reported herein. Questions which might be answered 
by this investigation are: 

1. To what extent have revisions been successful? 

2. What rates of progress within IMS are observed as 
students and teachers become more familiar with the 
program? 
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3. What changus in standardized test scores are observed 
for above grade level students as these begin to 
"catch up" to grade level topics within IMS? 

It may be noted at this point that questions regarding effective 
ness of changes in teacher training procedures will be investi- 
gated through a grant from the National Science Foundation. 

The primary purpose of this grant is to train approximately 
500 teachers for expanded use of IMS. Provision is made for 
evaluation of the training, especially with respect to 
questions raised in this report. 




Several areas of investigation were not possible during 
1970-71 due to lack of resources. These include: 

1. Systematic observation of classroom activities to 
determine proportion of time usage in various 
categories, such as waiting for materials, taking 
tests, working with learning pages, participating 
in activities, etc. Findings of disproportionate 
time usage could have implications for many aspects 
of the system. 

2. Study of individual uses of learning materials would 



show the relative effectiveness of various 
prescription and materials usage styles for different 
types of students. Such a study could lead to 
changes in materials and recommendations regarding 
prescriptions . 

Materials could be further upgraded by an observa- 
tional study of classroom difficulties encountered 
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by students. Teachers complained at times of the 
number of trivial questions asked by students who 
ha' trouble interpreting materials. While some of 
this problem may be endemic in all groups of 
children, certain materials in IMS may aggravate it. 
If these materials could be identified and revised 
the system might be improved substantially. 

The foregoing do not exhaust the possibilities but are 
presented to show the broad areas of possible further research. 
In some respects this research and effecting changes there- 
from will be increasingly difficult as IMS becomes more widely 
used. The cost of revision materials for 23 schools is much 
less than for 150 or 1500. Therefore, this report is concluded 
with a plea for support for continued research. IMS has shown 
itself to be a high quality, robustly successful system; it can 
attain even higher levels of excellence through systematic 
application of results reported herein and those from further 
research studies . 
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Appendix I 



Suggested Materials List for IMS 
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Suggested Materials List for IMS 



Materials have been divided into four classes: 

A. Supplies— includes general office and school supplies normally 
already included on school purchase orders. 

B. Math materials to buy— includes items that are specifically 
math oriented. 

C. Materials to make— includes cards used in activities. 

D. Materials to bring from home. 

Kerns are listed in alphabetical order. In some cases, suppliers are 
listed from whom the materials may be purchased. A list of suppliers whose 
catalogs will be helpful is included. Some materials listed under Materials 
to Make” may possibly be bought, while materials listed under ‘Math 
Materials to Buy” may be constructed. Starred items are considered 

essential to IMS. 
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A. Supplies 



Ball 

Calendar 

Chalk 

Construction paper 
Crayons or pencils 
Graph paper 
Gum labels 

Index cards (see Materials to Make) 
Magnet 

Magic Marker j 
Masking tape 
Mirror 
Newsprint 
Oaktag 

Overhead or Opaque Projector 
Paper clips 
. Paste 
Pipe cleaners 
Plasticine 
Rope 
Ruler 
Scissors 
String or yarn * 

Test tube 
Yardstick 
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B. Math Materials to Buy (incomplete in upper levels and geometry) 

* essential to IMS 
Abacus (suggest H & M Assoc. ) 

a ccobloc s (succest H & M Assoc. ) , . 

Attribute blocks— pattern blocks and colored string may be used. 

(suggest Webster Div# 9 McGraw-Hill) 

Automatic calculator (suggest SEE) 

Beads— to string (suggest Ideal) 

Building blocks (suggest Play Skool) 

Centimeter rods (suggest Cuisenaire) 

Meal, Milton-Bradley) 

Dice— or paint dots on cubes • 

Dienes blocks (suggest Herder & Herder) 

Drum and drumstick . x 

Flannel board or magnetic board (suggest Weal) 

Geoboard and elastics (suggest Cuiser^ire or H&M) 

Liquid measuring kit— should be plas* * and include cup, p , 
ouart, gallon (suggest REC) w 

Mirror cards (suggest Webster Div. , McGraw-Hill) 

^ a * • 



. £ ; • 



m — 













* 



(suggest SEE) . 

One-inch colored cubes (suggest McGraw-Hill) 
Pan-balance scale— suggest 



m 



In preference to 



3 



• (suggest SEE) 

Pattern blocks- (suggest SEE or McGraw-Hill) 

Play ^ney^toctod^ec^ins (suggest Meal, Milton-Bradley) 
Primary ruler (suggest SEE) 

Primary shapes (suggest H&M) . 

S&MfiS SKfflWVS- „ 
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Ci Materials to make 

The following cards are used only in activities. They may be made as 
the activities are used, may be pre-constructed, or in some cases, may be 
bought. The method used to make the cards would have an effect on the 
supplies list. Index cards, br heavy grade paper that is cut, may be used. 

CARDS: (suggest Ideal and Milton-Bradley) 

ADDITION-SUBTRACTION STATEMENT— problem statements indicated 
either by “+” or with or without sums or differences: 



6 . 2-*- 7 




6.2 
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ARROW CARDS— for use with 1-100 number chart: 

irn EEI] r~n~] 



etc. 



ASSIGNMENT CARDS— cards with two sets of dots* 
CLUE CARDS— use number 1-6 for each direction: 



l 0 l ooH 



r npl~| | down X j | left 1 | |. right l] I move 1 ) 



DAILY ACTIVITY CARDS— showing activities relating to specific days of 
the week, e. g. , church on Sunday, etc. 

DOLL CLOTHER DOT CARDS— numbers and sets to 10: 

ran 



i 



DOT CARDS— cards showing dots, one to ten: 

# 

r~o~i r°°~i i 000 ! 

FRACTION - CARDS— -including / 41 11/ |j IV 3| 0 
MONEY CARDS— cards showir f Quires of coins ana groups of coins. 
MULTIPLICATION COMMUTA1 ;E— front has: 

I Q (2*8) 



back has | 2(6’s) | 



etc. 



MULTIPLICATION-DIVISION STATEMENT— see Addition-Subtraction. 
MULTIPLICATION PICTURE CARDS— pictures in set groupings on front 

(: : :), with designation of sets of each on back. 

NUMERAL CARDS— one numeraTper card. 

Frequency of use: 

“ “ - " 71 times 
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numerals 1-10 
numerals 1-25 
nui erals 1-100 
numerals 10,20. • • 
ifcnd-up numerals 
(1-10) 



33 times 
10 times 
1 time 
6 times 

110 




ONE MORE-ONE LESS CARDS- L 1 I°5D U ^S SJ 
ONES, TENS CARDS— I ones 

OPERATION CARDS— 

| minus] 1 >1 GO 

ORDINAL CARDS— ordinal numbers, first-tenth* 

PICTURE CARDS— with circled subgroups 

; or word-situation problems 

PLAYING CARDS— regular cards with honors (king, queen, etc. ) removed. 
PUZZLE CARDS--pleces that flt together to match numeral with corresponding 



set pictures. 



HZ 
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1-10, .10.1.11, »*W«, 

SET-WCTTOE^ ARDS— sets showing numbers 0-10, with corresponding 
pictured shapes or objects. 



E 



STRUCTURED GROUP CARDS— shows structured groups to 25. 



ljcv^ v. i 



WORD-NUMBER CARDS— 1 one | « 



| two] $ through [ ten | < 



Additional materials to make. ' 

MOtre^^LANSPARENCIES— for overhead projector. Shows pictures of 

NUMBcTcARDS— similar to Bingo cards. JEfcd 

POCKET CHART— for display purposes. Library pockets may o g 

SHOW-M^?ARD^OCKET--consists of tagboard folded and stapled with 
room for three numeral cards to be displayed. 



pi . ^ , r» n 
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D. Materials to bring from home 



Bottles and jars 
Buttons 
Candles 
Clotiiespins 
Coat hangers 
Egg cartons 
Kitty litter . 

Magazines and catalogues 
Milk cartons 
Paper boxes or crates 
Paper cups 
Paper plates 
* ; Plastic eating utensils 
Sand 
Toys 
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SOURCES OF MATERIALS 

Cuiscnairc Company of America, Inc. 

12 Church Street 

New Rochelle, New York 10805 

Educational Development Center 

55 Chapel Street 

Newton, Massachusetts 02160 

Herder and Herder 
232 Madison Avenue 
New York, New York 

H & M Associates 
Math Media Division 
P;0. Box 1107 
Danbury, Connecticut 06810 

Ideal School Supply Company 
Oak Lawn, Illinois 60453 

Webster Division 
' McGraw-Hill Book Company 
Order Service Department 
Manchester Road 
Manchester, Missouri 63011 

Responsive Environment Corporation (REC) 
Learning Materials Division 
Englewood Cliffs, New Jersey 07632 

Selective Education Equipment, ($EE) 

3 Bridge Street 

Newton, Massachusetts 02195 

i 

Belsco 

5100 West 82nd Street 
Minneapolis, Minnesota 55431 
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Appendix II 



Supplies and Materials Questionnaire 
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SUPPLIES AND MATERIALS QlIKSTIONNA IRK 
SCHOOL 



SUPPLIES 






Please check (/) each of the following that is used in your school 
in connection with IMS. Put two checks (//) if you think an 
item is of particular importance or is used in substantial 
quantity. 



Ball 

Calendar 



Chalk 



Construction 



Crayons or pencils 



_Graph paper 
Gum labels 
Index cards 



_Magic Marker 



^Masking tape 



Mirror 



^Newsprint 



Oaktag 



jOverhead or Opaque 
projector 
Paper clips 



Paste 



_Pipe cleaners 
_Plasticine 
Rope 



Ruler 



Scissors 



String or yarn 



Test tube 



Yardstick 



Magnet 



PURCHASED MATERIALS 



Available Well used 



Item | Yes 


No 


Number 


Brand 


Yes 


No 


Abacus 














Attribute blocks 














Automatic Calculator 














Beads — to string 














Building blocks 














Centimeter zods 














Clock 














Counters 














Dice 
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Item 


Yes 


No 


Numb* : 


Brand 


Yes 


No 


Dienes blocks 














Drum and drumstick 












! 


Flannel Board or magnetic 














Geoboard and elastics 














Liquid measuring kit 














Mirror cards 














Number balance 














One- inch colored cubes 














Pan-balance scale 














Pattern blocks 














Pegs and pegboard 














Play money 














Primary ruler 














Primary shapes 














Stick counters 














Uni fix cubes 














Walk-on number line 















* essential to IMS 

Please list other purchased items below: 



! HOME MADE MATERIALS 

I 

I Please list below home-made or improvised materials such as 
I drill cards for number facts# fractions kits# egg cartons # milk 

I cartons# kitty litter, etc.# that are used profitably with IMS 

at your school. Continue on back of page. 
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Workshop Evaluation Form 
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IMS 

WORKSHOT EVALUATION FORM 



KSYs SA (Strongly Ag r*e), A (Agree) # 7 (Undecided), D (Disagree) , 
R (Strongly DisagreeT 



* 

r 



1. 


The objectives of this workshop were eleer to as. 


M 


A 


7 


D 


sr 


2. 


The objectives of this workshop were realistic. 


BA 


A 


7 


D 


rr 


3. 


The participants accepted the porpoees of this workshop. 


BA 


A 


7 


D 


iST. - 


4. 


The objectives of this workshop were the sees as ay 
objectives . 


BA 


A 


7 


0 


Si*. 


5. 


z have not learned anything new. 


BA 


A 


7 


0 


SO 


i. 


The material presented sensed valuable to ne. 


BA 


A 


7 


D 


Sb 


7. 


Z could have learned as ouch by reading a book. 


BA 


A 


7 


0 


3D 


S. 


Insufficient attention was given to problem which eight 
be encountered in installation of XUS. 


BA 


A 


7 


D 


SD 


9. 


The information presented was too elamntary. 


BA 


A 


7 


0 


SO 


10. 


The speakers really knew their subject. 


BA 


A 


7 


0 


SO 


11. 


Z was stisnlated to think about the topics presented. 


BA 


A 


7 


0 


SD 


12. 


Ms worked together veil as a group. 


BA 


A 


7 


0 


3D 


13. 


The group discussions were eseellent. 


BA 


A 


7 


0 


SD 


14. 


There was little tins for informal conversation. 


BA 


A 


7 


0 


SD 


IS. 


Z had no opportunity to express my ideas. 


BA 


A 


7 


0 


SD 


14. 


z really felt a part of this trap. 


BA 


K 


7 


D 


SD 


17. 


My time was well spent. 


BA 


A 


7 


0 


sr 


It. 


The workshop net my expectations. 


BA 


A 


7 


D 


S3 


19. 


Too much time was devoted to trivial emitters. 


BA 


A 


7 


D 


SD 


20. 


The information presented was too advanced. 


BA 


A 


7 


D 


"r 


21. 


Too such time was spent on theory. 


BA 


A 


7 


D 


r • 


22. 


The schedule should have been more flexible. 


BA 


A 


7 


D 


sr 


23. 


Objectives to be achieved were clearly stated at the 
beginning of the workshop. 


BA 


A 


7 


0 


"D 
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24. 


Objectives of the workshop were achieved. 


8A 


A 


? 


0 


80 


25. 


The presentations of the workshop coordinators were 
necessary for the achievement of the objectives. 


8A 


A 


? 


D 


SO 


26. 


Not enough time was spent practicing skills learned in 
the workshop. 

« 


SA 


A 


? 


0 


SO 


27. 


Materials provided were helpful in the learning of IMS 
procedures and principals. 


8A 


A 


? 


0 


SO 


21. 


In general# sessions held each day verui 













a* too frequent b) about right c) too infrequent 

29 . In general # the sessions held each day were* 

a) too long b) abcut right e) toe short 

30. The background you brought to the workshop as coopered to the amount of 
background presupposed by the presentations end source materials east 

a) anich greater b) some* that greater c) about the sane 

d) somewhat lass e) euch lass 

31. How many opportunities were there for you to interact with the instructors 
of the workshop to get individual attention? 

a) a great nany b) sons c) few d) none 

32. How nany opportunities were there for you to interact with the other 
workshop participants? 

a) a great nany b) sane c) few d) none 

33. To what extent do you feel that your understanding of behavioral objectives 
has been increased during the workshop period? 

a) a great deal b) some o) little d) none 

34. To what extent do you feel that your understanding of IMS has been increased 
during the workshop period? 

a) a great deal b) some c) little d) none 

35. hate yourself on the amount of confidence you have in being able to teach 
the IMS mathematics program in your school. 

little much 

36. The objectives o£ the workshop could have been attained best by t 

a) present method b) more lectures c) more prograaned material: 

37. if you had to do it over again* would you participate in this workshop 
you have just completed? 

•) ires b) no c) not sure 

3t. if a workshop such as this is held again# would you recosnend to others 
like yourself that they attend? 

•) yes b) no o) not sure 

39. In terms of learning the content of this session# the length of this 
workshop period west 

a) too long b) about right el too short 



O 



1.19 



m / tmet m m n** 



To what extent vara tha objectives of this workshop attained? 



What other objectives should be included for future workshops? 



Xn your opinion# what vara tha aajor strengths of this workshop? 






Xn your opinion# vhat vara tha najor weaknesses of this workshop? 



Additional c peasants about this workahopt 



i 
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INSTALLATION MONITOR REPORT* 



SCHOOLS PRINCIPAL: 

CITY: DATE: 

PURPOSE: 



I. Observations: 

A. Teachers 



B. Aides 



C. Students 



i 

! 

i 



D. Materials 



E . General 



II. Suggestions to RELCV: 
A. 



O 

ERJC 



* Subject to further development 
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page 2 



II. Suggestions to RELCV (cont.)i 
B. 



Monitor 






i 



o 

ERLC 
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in:: survey #i 



NAME TOTAL RESPOM DAMTS « 178^ 

sciioui. 

* 4 1st grade 33 3rd grade 35 5th grade 4 no grade 

Gi'AJE 38 7nd grade 4 0 4th grade 24 6th grade given 



• of Omissions 



PLKSOriAL DATA 



1 . Are you using IMS in your classroom now? Yc» l7SN o 1 Responses here 

after are for 

2 . Total years of teaching exoer lonco i 10.8 average the 175 "yes" 

respondents to 

3. Prior experience with IMS: <1 Yes 941 Mo question 1. 



4. Prior experience with IPI or other individualised mathematics system: 

resJ,li_No_i2» 

5 . Highest degree held: Bachelors na M asters i 7 t Doctors 



4. Background in RMthesiatics: 

1 • below average 

Average for an elementary teachef 2 • average 

1 • above average 

Above average average response • 2.20 



Below average 

7 . Do you feel your math background is satisfactory for teaching with 
IMS? Yes ass Mo as 

8. Did you attend an fils Workshop? Yes ass M O ut 

If your answer to #8 is "No,” how many hours of formal training in 
IMS procedures have you had? . f 

9 . Do you have a teacher aide for work with IMS? Tes oaff o 1 ■>» 

If your answer to *9 is "No,” and you feel you need one, please list 
the duties you think the aide should perform: 
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IMS GOALS 



I.-lb has Uun developed to achieve a number of goals. Some of these 
arc listed below. Please indicate whether or not IMS has achieved these 
goals, based or your experience to date. It is understood that your 
opinions may bo only tentative at this point. You may omit items if 
you ha"c no basis for giving an opinion. 

(3) i vhc learning materials are attractive. 

100%A gree 

Disagree 

Suggestions for improvement : 



(19) 2 . The use of graphics makes IMS interesting to pupils. 
98% A gree 
2% D isagree 

Suggestions for improvement: 



3 , The availability of concrete, pictorial, and abstract presentations 
(18) for teaching the same objectives permits the accommodation of indi- 

vidual differences. 

97% A gree 

it Disagree 

Suggestions for improvement: 



0 

ERIC 
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(5) 4 1 li injoy working with IMS materials* 

J 9% 'lev 

_ 1% L)i 'ogree 

Sag test ions for improvement: 



(18) 5. The teaching aids are attractive. 
99% Agree 
1% Disagree 

Suacn.sl: n* for improvement: 



(34) The provision of activities as well as work snouts permits me teacher 
j to assist the pupil in achieving curriculum goals. 

. B6% Agree 

^4% D isagree 

i 

; Suggestions for improvement: 

I 

I . — • ■ u — — — — - — 



(5) ^ - IMS teacher training materials are effective. 

86% Agree 
JJ1P isagree 

Suggestions fer improvement. 
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(17) a. .i. li ply of IMS nutonall in my school ib advqu »tr to moot pupil 

«i'.- 1- . 

51% <vj co*i Suggestions for improvement : 

49% ;?;igree 



(11) if. r-Jo materials are sufficiently durable to be reusable. 

97%Agr oe Suggestions for improvement: 

3%r>isngree _ 



1 



i 

i 

I 




(8)10. IMS laminated materials are easy for pupils to use. 



95%Agree 



Suggestions for improvement: 



5%Disagree 



(5111. Pupils in the higher grades can assume many of the responsibilities 
which normally require the assistance of a teacher aide. 



91% Agree 



Suggestions for improvement: 



9%Disagree 



(2212. IMS enables the teacher to spend more time giving individual mathe- 
matics assistance to pupils. 

72% Agree Suggestions for improvement: 

28% Disagree — 



1 



i 

; 

* 

I 






.. . ! 



lJuaiiaa 
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(4) 13. Pupils are able to scoru accurately their own tests. 



12% Agree grades 

1-3 

88% Disagree 



<2% gradas Suggestions for improvement: 
4-6 

38% 



(19)14. Pupils clean and replace the materials without assistance from the 
teacher . 






83% A gree 
17% Disagree 



Suggestions for improvement: 



(9) 15. 



IMS procedures for keeping track .of pupil progress provide the teacher 
with an accurate picture of each student's achievement • at any point 
in time. 



97% A gree 
3% Disagree 



Suggestions for improvement: 



16 The training you received prior to starting the school year provided 
a sound basis' for using IMS in the classroom. 



63% A gree IMS work- 
shop attend- 
37% Disagree ees- 



41% non Suggestions for improvement: 
attend- 

59% ees — 



(72) 17. IMS is more effective than the system you used last year. 

69% A gree Sugrestions for improvement: 

31% D isagree - — — — — 



O 

ERIC 
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IMS Problem 

ir V"'. '"Vr no for in opinion on an item, simply omit your respond 

1. * ' » < inont Tost is noedfri for Level 1 to detorminc whether students 

nh owl horun with Level II. 

80) _8 4 % Agree 

_16%' • Mqroc 

2. A '‘retest is needed for Level I to permit more accurate prescription 

writing in Level I. F F 

1) 91» Agree 

_£1 D isagree 

3. What is the prevalent student opinion of IMS? 

l) 98% favorable • 

2% unf avorable , 



). What is the prevalent parent opinion ot IMS? 

9) 73% favoraV»1» 

27% unfavorable 

1 To what extent do parents understand IMS reporting of student achievement? 

32* A lmost all understand 

6ft* A substantial number do not understand 

. Do measurement activities involving use of string, hands, toothpicks, etc, 
work out well? 

3ft* Yes 

69* Mo 

Do carts serve adequately for filing and keeping track of skill folders? 

(this response counted as an omission) 



130 



on* ^’es 
lf>% No 

We do not use carts 
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. ; .k.li ; jlrkr nc-od to o« repel. >t«U lull »iro. 

• 1) 741 vi.c 

261 u i .. a; r co 

■i. jjul 4 aomo children in your class profit more from some math progi im 
Oti.o i‘ than IMS? 

471 Yd & If yes, please describe charactenst i-:. 

531 no Almost ell •‘yea" respondents cite d 

difficulty end lack of drill for low ability students. 



I 



33 ) 

36 ) 

39 ) 

111 ) 



lu. Check the math program which: 

(a) Is oasoier for tho teachor to work with in the classroom 

261 IMS 74 E onventional 

(b) Requires less planning or other outside preparation by the teacher 

47 % IMS 53 % :onventicnal . 

11, * Placement tests should not be laminated. 

50% A gree 
SOI D isagree 

12. To what extent should placement tests be given in second grade ? 



They should be given in all areas 1 * all areas 

2 * some areas 

3 - no 2nd grade placement 
average response = 1.25 



No Placement Tests are needed 



They should be omitted in some areas when children have no prior IMS 
experience and all are likely to fail. 



List areas: 



13. Are lower level skill folders satisfactory for older children who 
need remedial work? 

75) 91% Yes 

9% No If no, list skills that are noticeably 

bad in this respect (for example., Numeration II, Skill 2): 
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1). nu. » t to "curved numbur line* approach in lime, Laval It work «>u 

wo 1 1 ? 

( 118 ) 42 %_Vl. . 

130.*' 

Ij. what Kind of writing inatriunant for tha laminatad shaats do you lik« 
Oast of those you hava triad? (Please giva brand naam and model or 
identification number.) 

16. What ia your biggest problam with IMS? 



Teacher * find thwlr Him mn fully occupied hy answering procedural 
auaatin ni that they in imahla frn glue individual inatruction on 
mathematical prnhlui. (Tr <« hoped proKlam will auhalde ae 

children and teachers become more familiar with IMS.) 

MATERIALS REVISION 



It ia our plan to reprint and rediatribute during the course of the achool 
year some of the skill pages or other materials that need revision. 

Please list below no more than ten skill folder pages or tests that need 
immediate revision. These should be the ones that cause the most trouble 
during the math period. 



1. 



2 . 



3 . 



4 . 



5 . 



6 . 



7 . 



8. 



9. 



10 . 



0 
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IMS SURVEY #2 



Name 

School 

Grade 



Please answer all questions for which you have sufficient information. 
Some questions are similar to those of the previous survey in order 
to permit changes of opinion and responses by those who omitted 
earlier. 



1. Are you using IMS in your classroom now? 
Yes 



No 

2. Did you attend one of the regional IMS Workshops in 

V7inston-Salem, Chesapeake, or Columbia held during the 
summer of 1970? 



Yes 



No 



t . 3. Did you- have formal • training or experience- in IMS procedures 
| other than that covered in 2 above? 



Yes, training conducted locally the* summer before or 
charing- the school year- by non- IMS personnel- How many 
hours of- local training? 

Yesy experience and/or training from the prior school 
~ year. 

Yes, other. Please explain: 




No formal training or experience prior to using IMS 
this year. 

Was -the- preparation you had prior to . teaching with IMS this 
year adequate? 



Yes 







, 1 , 



p lease check below .my 
not adequately supj 



UulN ot !«•»»•■« Ills JiM-li I'CCH Cl 

i *•■ 1 : m y * >ti t . -it -c 1 . 



In short Previously in 

s cipply now short supply . 



Special por.s or ponciis 
Skill fnliers* 



Manipulative materials 

•Please list below the skill folders you find should be stocked 
more heavily in the carts: 



6 . 



In some cases, supply problems a r.? 
conditions. Please check if either 
problem areas in your school. 



related to local storage 
of the following are 



Lack of carts or storage apace has caused problems. 



School layout prevents efficient use of available 
"materials . 



7 . 



n your grade, approximately what percentage of _the children 
;ould score accurately their own check-up tests. 



0 - 25 % 25 - 50% 

50 - 75 % __75 - 100 % 



8. Do you encourage or require 
children to do any of the 
following tasks? 



Approximately what 
percentage of your 
chi ldren - actual ly do 
these- tasks with 
reasonable success? 



Yes 



No 



ERjt 



% successful 



Score skill pages 
Score check-up tests 
Score pre-tests 
Write prescriptions 
Obtain, arid return materials 



‘MS’ 



- 1 - 



•i , t*i(, j 1 1 • *_ fit- to* i u* in«t !'C *» r i.ii-j! 'it> tv i > iLu.ic.it opinion of f i-’ • 

F«» • iM • 

i i *.», • 1 



I Hi I . I Vi > i . 1 1 > I ** 



10. i ji.'t, *. t the following best '..J. o <> t.« 

Favorable 

Mixed 

Unfavorable 



, • . nt * jiinion > t IMS 



11. To what: extent do you think parent s undo: :;U*.u i certain features 
'f. IMS? 

Almost all Many do nn<- 

i in do s r, t. and understand 

. i., i/... | i r . yi:;; af‘ hi:*, own level. 



Kiel) t udent. works at his own speed. 

Completion of a level represents 
mastery* not exposure as in the 
textbook approach. 



12. Do teachers need to keep a separate set of Guidelines at 
their desks? 



Yes 



No 



13. Should keys be laminated? 



Yes 



No 



| 14. Do you find IMS materials generally effective for helping 

students .achieve- the’ IMS Behavioral Objectives? 

Yes,- with the possible- exception- of: *Mu-ltiplication I 

Skill- 2; Multiplication* II, Skill 6; Measurement II, 
Skill 1; Division II, Skills 2,3; Time III, Skills 2,3. 



ERIC 



NO 

♦These skil l- folders will undcrco extensive revisions. 

- ' m 






4 - 



Tf ol-lior skill folders have pvov»v« 
] . I e < i s u I i a t: t : h om h o r e : 



t > '» r i irnlnrly unsnt i .s factory 

i 



I,o vo I Strand £?k LLl / 

— ( 



Does provision of activity pages in addition lo- work pages 
assist in achievement of IMS Behavioral (..»bie<;l:.ive » i 



Yes 

No 



Does IMS enable the teacher to spend invio 
assistance to students? 

Yes 

No 



i ] iv ing j.nd i vidua I 



Does IMS seem more effective than* the mathematics- system you 
used previously? 

Yes 

No 
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IMS TEACHER SURVEY # 3 



May 21 , 1971 



PLEASE DO NOT OMIT ITEMS. 

If you are not sure of a response, mark the choice which seems 
closer to your belief. 



1. The learning materials are attractive. 

99% A gree Suggestions for improvement: 

Disagree 



2. The use of cartoon characters and other drawings makes 
IMS interesting to pupils. 

98% A gree Suggestions for improvement: 

Disagree 



3. The availability of concrete, pictorial, and abstract 

presentations for teaching the same objectives permits the 
accomodation of individual differences. 

98% Agree Suggestions for improvement: 

Disagree 



Pupils enjoy working with IMS materials. 

99% Agree Suggestions for improvement: 

Disagree 



m 



4 . 






[ 2 ] 



r ). 



oblrH!!o PWlat i VG materials assists pupils in mastering 
objectives beyond achievement based on skill pages alone'. 



93% Agree 
Disagree 



Suggestions for improvement: 



6 . 



The provision of activities assists 
objectives beyond achievement based 



pupils in mastering 
on skill pages alone. 



9 2 % Agre e 



NON- USERS EXCLUDED 



Disagree 



101/1,96 M y students have not used activities extensively 
enough to permit judgement. 

Suggestions for improvement: 52% of responders had not 

used activities to any extent. 



IMS teacher training materials used in IMS summer workshops 
are effective. r 

^^ 4/196 1 ^1^ n °t attend an IMS summer workshop 

— Agree Suggestions for improvement: 

Disagree NON-ATTENDEES EXCLUDED 



IMS User Guides are effective. 

Agree NON- USERS EXCLUDED 

Disagree 

32/196 User Guides have not been readily available in my school. 

Suggestions for improvement: 16% of responders in this 

category. 

140 “ 



IMS materials are sufficiently durable to be reusable. 

9 2% Agree Suggestions for improvement: 

Disagree 



IMS laminated materials are easy for pupils to use. 
9 6 % Agree Suggestions for improvement: 

Disagree 



IMS enables the teacher to spend more time giving individual 
mathematics assistance to pupils. 

84% A gree Suggestions for improvement: 

Disagree 



Grade taught 



(We recognize that the grade taught 



will have a bearing on your responses to questions 13 - 17.) 
— -? 3 4 5 6 OMIT TOTAL 

15 3 7 44 38 38 22 T 196 

Almost all the pupils in my class obtain , clean/ and return 
skill pages with very few errors. 



71% T rue Grades 1 * 3 
False 



81% 



Grades 4-6 



Half or more of my students write (or could write) their 
own prescriptions. 

22% True Grades 1-3 81% Grades 4-6 

False 
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15. Half or more of my students can (or could) score their 
own skill pages accurately. 

Tr ue Grades 1 - 3 87% Grades 4-6 

False 



16. Half or more of my students can (or could) score their 
own check-up tests accurately. 

40% True Grades 1 - 3 75% Grades 4-6 

False 



17. Half or more of my students can (or could) score their 
own pretests accurately. 

10% True Grades 1 -3 64% Grades 4-6 

False 



18. IMS procedures for keeping track of pupil progress provide 
the teacher with an accurate picture of each student's 
achievement at any point in time. 

94% A gree Suggestions for improvement: 

Disagree 



19 Please compare IMS with the usual textbook based approach 
to mathematics instruction. Check the one which you think 
is more effective for the following: 

MORE EFFECTIVE 
IMS Text Book 



Teaching of concepts 76% 



Teaching of skills 71% 



Teaching of applications 80% 



Teaching high ability students 84% 

Teaching low ability students 72% 



I Teaching medium ability students 83% 

me 142 
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19. (cont.) ±22 — 

Motivating students JH% — 

Accomodating individual differences 98% — 

Arranging coverage of needed topics 79% 

in mathematics - — 

20. Considering all aspects of effectiveness, 
listed in 19 and others you may think of, 
do you consider generally more effective? 

91% IMS 

Text Book 
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Text Book 



both those 
which approach 
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INCIDENT REPORT* 



SCHOOL: 

CITY: 



PRINCIPAL: 

DATE: 



I. Observation: 



II. Suggestions to RELCV,: 



Teacher 



* Subject to further development. 
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EC QUESTIONNAIRE 3-1-71 



j 



Name 

School 



Please list the number of classes and approximate numbers or 
students in IMS in your school as of March 1, 1971. 

Grade 1 2 3 4 5 6 

No. of Classes 

Appr. No. of Students 



We will soon print and distribute replacements for certain 
skill pages. Please count the number of folders in your 
school as listed below so that we can send the correct 



numbers cf replacements: 



Level 


Area 


Skill 


II 


Numeration 


8 


II 


Division 


2 


II 


Measurement 


1 


IV 


Addition 


1 



3. We need to have a fairly precis 
are used in your school. Pleas 
questions do not seem to permit 
of your aide situation. 

PAID AIDES 



Yes Do you have paid IMS 
aides? (If answer is 
No no, omit questions 
below.) 



Yes Do paid aides work 
directly with stu- 
No dents during math 
period? 



No. of Folders 



I 

l 



picture of how aides (if any) 
explain below if the following 
an accurate characterization 



VOLUNTARY AIDES 

Yes Do you have volunteer 
IMS aides (parents, 

No high school. or college 

students)? If answer 
is no, omit questions 
below. 

Yes Do volunteer aides 
work directly with 
No students during math 

period? 







- 2 - 



Y ( ’ 



No 



Yes 

No 



No. of 
min . 



No. of 
min . 



Total 

full- 

time 

IMS 

aides 



I 'AH' A. I Dl 



A i < > tv) ill • i i 1 1( u: • u. illy 

) > ro'W’ii I; iii I lu • I M.i • 
room during tin- math 
porj.od? 

Do paid aides score 
tests, record progress 
or do other clerical 
jobs within the 
system? 





VOLUNTARY A 


i ni: 


i ^ 


Yes 


An- w lun 




1 . 1 i 1 1 1 ’ : 




usual. 1 y p 


1 i • : • 


i'p i i n 


No 


the IMS r 


oom 


duri nq 




the m«:; t!i 


iHvr 


led 


Yes 


Do volunteer 


aides 




score tes 


t & 


record 


No 


progress 


or 


do other 




clerical 


job 


s within 




the system? 





For the average 
teacher approximately 
how much paid aide 
time is given for IMS 
each day? 

Apnroxi matol.y how much 
1 on go r wo u 1 d t h e 
average t.O'Vi'or have 
to work each day it 
p a i d a i. d o ; ; w e r e w i t h - 
drawn from .IMS work? 



For the average teach- 
No. of er approximately how 

min. much volunteer aide 

time is given for 
IMS each day? 

Approximately how much 
No”i of longer would the 

min . average t.eacher have 

to Work each day 
if vo 1 un t eer a ides 
were withdrawn from 
IMS work? 



How many full-time 
paid IMS aides does 
your school have? 

(Do not report aide 
usage on non- IMS 
work. Total need 
not be a whole 
number, e.g., 3 3/4 
full-time aides 
might represent 2 full- 
time on IMS, 3 half-time 
on ikS and one half-time 
worker used only 50% on 
IMS . ) 



Other comments or explanation of aide usage: 



i 
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4 . 



Arc tiny classes ill your school not nntor ini) IMf> fhi.s> V ( 'ni" 
clue to problems connected with availability of materials? 

Yes 

No 

How many classes? 

If "yes," please tell how many classes are affected by each of 
the following problems. 

Enter number of classes affected. 

i School layout or floorplan requires additional materials. 

7 Mot enough carts (or shelves) are available. 

| Level VII and hiaher materials are needed. 

i — 

Manipulative materials are needed. 

| Heavy concentration of students at lower levels requires 

more materials. 

I If the above does not account for all classes not in IMS due 

I to materials problems, please explain below: 



EKLC 
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March 31, 1971 



Report of 



A Review of the Individualized 
Mathematics System (IMS) 

by 

Robert M. Gagne 
Florida State University 



Submitted to the Regional Education Laboratory for the Carolinas 
and Virginia, Chapel Hill and Duke Streets, Durham, North 
Carolina 27701. 




A Review of IMS 



i 



Scope of Review 

This review has the Intention of examining and commenting on IMS as 
a system for learning . Having this purpose, the reviewer has been par- 
ticularly concerned with the following aspects of the system: 

1. The learning objectives of the system, and their ordering from 
level I to IX. This examination has been conducted by reference 
to the list of behavioral objectives, and the representation made 

of them in the Guidelines and Skill exercises. A sampling technique 
was employed here. 

2. The ordering of topics within each skill level. Again, the objectives 
were examined, and sampling was used to gain an understanding of 
their meaning in operation at various skill levels. 

3. The kinds of pupil activities as represented in skill sheets and 
activity sheets, the manner of presenting them, at various levels 

and areas. 

4. The testing-prescription cycle, as described in user guides and as 
illustrated by pre-tests, direct tons for prescription, post-tests, and 
skill sheets. 

5. A consideration of the rationale for learning provided by the IMS 
system. 

It should be apparent from this list that there are definite limitations to 
the scope of this review, which arise both from restrictions of time and from 
limits on the capability of the reviewer. It is assumed that these other aspects 
of the system may be otherwise evaluated, either by examinations of the system 
by specialists with different capabilities, or by the use of empirical data obtain- 
ed from measures of pupil performance, or both. To be specific, this review- 
er has not attempted to draw inferences or conclusions about the following 
aspects of the IMS: 

(1) the precision or validity with which the mathematical content is 
represented; 

(2) the logical defensibility within the domain of mathematics of the order- 
ing of topics which relate to each other (such as addition and subtrac- 
tion, or multiplication and division); 
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(3) the existence or desirability of alternative methods of per forming 
mathematical operations; or 

(4) the evaluation of goals in elementary mathematics reflected by the 
totality of objectives, levels I through IX. 

Learning Objectives and their Ordering 

Two major characteristics of the IMS can be seen by examining the lists 
of objectives throughout nine levels, and comparing these with their represen- 
tations in skill sheets and pre- and post-tests. First, there is the matter of 
whether the objectives are stated in such a way that the/ communicate (primarily 
to the teacher), the purpose of the learning to be accomplished. Second is 
the question of how these objectives are ordered, and the extent to which this 
ordering represents a reasonable approach to pupil learning. 

Communication by objectives . In general, it is apparent that a serious 
attempt has been made to state objectives in objective, communicable language 
referring to pupil per formance. These statements usually communicate well, 
and appear to do the job of making clear to the teacher, and through her, to the 
pupil, the nature of the task to be learned. In the operation of the system, the 
objectives appear on the Guideline sheets, and the teacher can then make an 
immediate comparison of the skill exercises which correspond. 



The communicability of the objectives does vary somewhat in effectiveness, 
and many of them could be improved if it were important to the system for this 
to be done. From the standpoint of statements which convey an immediate 
impression, by themselves, of what the corresponding skill exercise is like, 
many of them leave something to be desired. For example, an objective such 
as the following does a pretty good job of immediately defining the necessary 
skill exercise (II Addition, 3): "Given the words 'plus' and 'equals', matches 
them respectively with the symbols '+' and , and vice versa". In contrast, 
the following statement does not provide adequate information (II Addition, 1): 
"Writes the cardinality of each of two sets and the cardinality of the two sets 
combined (to a sum of 18)". This statement is in a different form; it does not 
clearly identify the stimulus situation; and its verb does not clearly specify the 
behavior (as distinguished from the response). An alternative statement, 
avoiding these difficulties, would be: "Given pictures or objects representing 
two sets and their combination set, identifies the cardinality of each set by 
writing a numeral". 

Many other examples could be given of relatively good, and relatively less 
good, communication by these statements of objectives. I shall not try here 
to convince by piling example upon example. I simply want to say that communi- 
cability of these objectives is improvable, if it is important to do so. Iam 
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less sure that Lt Ls important to undertake such improvement, because it 
seems easy enough for the teacher to look at the skill exercise in order to 
confirm the meaning of the objective. For those who wish to prepare items 
wnich test the capability to be learned, it is less; easy. My suggestion 
would be that all objectives be written in a standard format, using a standard 
set of major verbs. The format is: (1) Given (the stimulus situation); 

(2) the major verb identifying the kind of intellectual processing required; 

(3) the object of the verb, including the mathematical purpose of the objective; 

(4) a gerundive phrase denoting the response; (5) limitations on the scope, if 

any. Using such a scheme, II Numeration 8, for example, would be written 
as follows: "Given two numerals representing whole numbers, identifies the 

relationships of equality, greater than, and less than, by placing ~> , * > ^ 
between pairs of numerals". 

Ordering of objectives . When one examines the sequence of objectives 
within a given area, throughout the nine levels, it is apparent that there are 
many progressive sequences, and that they make generally good sense so far 
as learning progressions are concerned. For example, it is clear that VI 
Subtraction 1, "Computes the difference of ,‘wo whole numbers requiring re- 
grouping (minuends to 1,000)" has been preceded by a sequence of skills in 
V Subtraction, which includes subtracting with regrouping involving two and 
three-digit numbers (3, 4, 5); subtraction without regrouping of three- and 
four- dig it numbers (2); and that these in turn have been preceded by mastery 
of subtraction facts through 20 (1). One can follow the sequence further to 
IV Subtraction, where the tasks are solving two-digit subtraction problems 
(2), and other operations requiring checking (1, 3). The concrete conceptual 
bases for these subtraction operations are established in III Subtraction, 

(1, 3, and 4), and earlier still in II Subtraction (3, 4, 5), and the concepts 
of "taking away" (2, 3) and "one less than" (4) in I Subtraction. 

I have sought out similar sequences in the other skill areas, including 
addition, multiplication, division, fractions, and so on. It is clear in each 
case that the sequence makes rational sense as a learning progression. This 
is true also when one examines some "subordinate themes" not necessarily 
in the mainstream of objectives, such as equalities and inequalities. For 
example, V Mixed Operations 3, asks the student to identify equalities and 
inequalities for expressions involving addition, subtraction, multiplication, 
and/or division. These operations have, of course, been previously taught 
separately. But in addition, one can also trace a progression of the concepts 
of equalities and inequalities through IV Addition 3, IV Subtraction 4, IV 
Multiplication 1, III Numeration 8, III Addition 2, III Subtraction 1, II Numer- 
ation 8, II Addition 3, II Subtraction 2, 6, II Multiplication 4, I Addition (1,^2) 
and I Subtraction (1, 2, 3, 4). In other words, there is evidence that learning 
sequences have been carefully planned throughout the IMS continuum, and this 
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appears to be true whether one traces them out within a given 
for what may be called "subordinate themes", among various 



skill a roa, or. 
skill areas. 



The question as to whether the ordering of skills to be learned appear., 
correct can therefore be answered quite confidently in the affirmative. A 
much more difficult question, however, is whether when viewed from the 
standpoint of efficient learning hierarchies, any skills have been omitted, hi 
fact, within the time available to me, I cannot adequately answer this question. 
As an example of what would be required, I select VI Fractions 4, "Finds the 
L. C. D. for a given set of fractions". 



The subordinate skills required for this activity would appear to be the 
following, which could readily be represented as a hierarchy (c.f. , Gagne, 
1970): 

1. Given whole numbers, identifies the factors yielding each as 
a product. 

2. Divides whole numbers (to 999) by small prime numbers to obtain 
factors. 

3. Obtains product of successive multiplication of small whole 
numbers (for example, 2x3x5=?). 

4. Checks results of division by multiplication. 



These subordinate skills are, in fact, represented in various subdivisions of 
the IMS Continuum. For example, identifying the factors of whole numbers 
occurs as VI Multiplication 6j dividing whole numbers as VI Division 2j obtain- 
ing products of successive multiplication as VI Mixed Operations 3; and 
checking results of division by multiplication as VI Division 4. 

Thus, it is clear from this example that the necessary subordinate skills 
for learning a particular skill are represented . It is notable, though, that the 
sequence for their learning is neither stated nor implied, and that as a conse- 
quence any given pupil may come up with "missing" subordinate skills. It may 
be noted that both the target skill and its prerequisites, identified in the pre- 
vious paragraph, all occur at Level VI. A rough sequencing, designed to 
insure that subordinate skills were mastered before superordinate ones were 
presented, would place these subordinate skills at Level V (or, alternatively, 
the objective VI Fractions 4 at Level VII). A more precise sequencing 
would probably require a rearrangement of many of the specific subordinate 
skills in the Continuum, as well as a different structure for pre-tests (to be 
discussed later). However, at this point it may be noted that no obvious 
omissions of subordinate skills have been discovered by a sampling procedure. 



Kmp i i-u al 'Evidence ol' Skill Dependencies 

The data obtainable from student progress and test records should pro- 
vide a rich source for the conduct of additional formative evaluation ol IMS, 
The developers of the system are doubtless aware of this point, but I point 
it out here with the intent of providing some additional emphasis. 



What must be sought in the examination of such data is not simply 
"difficulty", but evidence of dependency. The procedure may perhaps best 
be described abstractly. Suppose that a "normal" sequence has been hypothe- 
sized as Skill A- -Skill B- -Skill C. Of 100 children who have attained Skill 
A successfully, 60 are able to accomplish Skill B on their first try, whereas 
40 are not. Out of the 60 who are successful on Skill B, 54 (90%) are able 
to attain Skill C on their first try, and 6 (10%) are not. Of the 40 children 
who have not learned Skill B, 2 (5%) are able to attain Skill C on their first 
try, whereas 38 (95%) are not. These results indicate the dependency of 
Skill C on Skill B. To repeat the contrast that has been illustrated: of those 
who mastered Skill B before attempting Skill C, 90% succeeded; of those who 
did not master Skill B before attempting Skill C, only 5% succeeded. These 
two percentages differ markedly, and one is justified in inferring dependency 
of C on B. In contrast, if the two percentages found were, say, 52% and 
47%, the evidence for dependency would be at most exceedingly weak. 

Of course, the hypothesized dependencies (sequences) may sometimes be 
more complex, since a given skill may be supposed to depend on prior learn- 
ing of two or more subordinate ones. The basic logic, however, remains the 
same in such instances. The possibilities of determining efficient sequences 
of mathematics skills by means of such analyses of data would appear to be 
highly promising. 

Activities and Skill Sheets as Vehicles for Learning 

This reviewer has examined the Skill Sheets and Activities for a variety 
of learning objectives throughout the continuum of areas and levels. Such a 
review, of course, dees not make possible detailed judgments of which are 
good, better, and best. Presumably, evidence of this sort may best be ob- 
tained from direct evidence attending the use of the materials, such as 
teachers comments, student scores, etc. 

On the whole, the learning activities embodied in this system appear to 
be remarkably good. Their advantageous features include the following: 

1. The skill sheet exercises are generally clear, and the pictorial 
and diagrammatic features contribute much to this clarity. 
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As a consequence, it seems likely that: those cxori-iscs requi 
minimal verbal directions to the pupil. .This advantage is oJ' 
ticular consequence to a system of individualized instruction. 
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3. Generally, the pictorial aspects of the skill sheets command 
attention and interest. 

4. There is considerable variety in the specific representations of 
skill problems. A respectable amount of evidence shows such 
variety to be facilitative of learning and transfer. 

5. The skill exercises appear to be carefully matched to the objectives; 
that is to say, they are valid in content. 

6. The Activities are, generally speaking, imaginative and varied. 

They provide considerable flexibility to the teacher for the conduct 
of classroom activities. 

One characteristic of the Activities may deserve further comment. Generally 
speaking, they appear to be representations, often in the form of group activities, 
of the same learning reflected in the skill sheets. They may thus provide 
additional opportunities for learning and review. This is, of course, one 
possible use of Activities , and it may be the most important one. Another 
possibility which plight be given further consideration in future development 
is the use of Activities to promote transfer and problem - solving . Not many 
of those currently existing could be considered to fall into such a category. 

Such Activities would be designed deliberately to be "mind-stretching", to 
permit various kinds of unanticipated outcomes, and to emphasize applications 
of mathematical operations. They would, in short, more frequently require 
the generation of solutions to novel problems by students. 



Tests and Testing 

Tests for this program appear to be carefully designed. The items of 
Placement Tests, Pre-tests, and Check-tests are generally clear and easy 
to understand. They are varied in content, and their diagrams and pictures 
are appealing and helpful. 

The system for testing, as described in the User Guide, Volume 2, is 
highly reasonable and systematic. In practice, it may well result in the im- 
pression of "too much testing". Overcoming this kind of difficulty might be 
accomplished by various procedures having the aim of making instruction 
more like testing and testing more like instruction. In fact, these elements 
are difficult to distinguish. Other specific ways to reduce the impression of 
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"too miifh testing" imhide (J) reducing the frequency of use of Placement 
Tests, when the position of the student is evident from other measures: and 
(2) reducing the frequency of usage of the Post-test, when it is judged that 
its function can be served by Check- tests. 



It should be realized that I am not able to make specific recommendations 
about reductions in the frequency of "testing", since the major evidence of 
its desirability must c me from experience with the use of IMS itself. How- 
ever, the practical pressures for cutting down on the time for testing may be 
considerable in a program based upon such a systematic procedure for assess- 
ment as is IMS. Should such pressures be acceded to, it is obviously desirable 
that this be done in such a way that the basic purposes of assessment not be 
sacrificed. 



The Testing- Prescription Cycle 

As has been previously stated, the cycle of assessment and prescription 
is a highly systematic one, and the benefits of such procedures in improving 
student achievement should become apparent in those schools which use the 
system. The testing-prescription cycle has the evident purpose of making 
it possible for each pupil to begin learning new skills at a point which reflects 
his previous learning, to demonstrate the attainment of these new accomplish- 
ments, to review them when necessary, and to proceed to acquire other 
skills which advance his level of achievement in a planned manner. Basically, 
it is a system which bases advancement in skill on prior accomplishment 
(mastery) of prerequisite skills. 

What happens when a pupil is assigned the task of learning a new skill, and 
fails to do so? According to User Guide, Volume 4, the teacher may assign 
alternative work pages, or perhaps accompany these with some tutoring. Here 
a specific limitation of the system becomes apparent- -the extent to which pre- 
cise diagnosis is possible with IMS. I mention this limitation because it is 
of particular interest to me. It needs to be recognized, I believe, although 
any revision of the system to remove it would probably have to be rather ex- 
tensive. Accordingly, whether such revision should be undertaken would 
have to consider costs as well as possible benefits. 

The crux of the limitation in the diagnostic feature of IMS is this: there 
is a lack of specificity in proceeding from (1) what the pupil doesn't yet know 
how to do, as revealed by a Pre-test, and (2) what he should do to learn it. 

The Pre-test itself does not provide information which is diagnostic, in a 
precise sense, of what is needed for the pupil to attain a skill he doesn't yet 
have. It is conceivable that in some instances he needs nothing more than 
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verbal directions, whereas Ln others, an entire set of subordinate skills may 
need to be learned. 

Let me illustrate this point with an analysis of the third item, first page, 
of II Numeration, representing Skill 1. Suppose it is found that the pupil can- 
not do this task, "Write the number word, then match with the picture." One 
cannot tell from this item by itself whether he (1) fails to know the name of the 
numeral; (2) cannot write the name of the numeral; (3) cannot identify the 
numbers describing the sets of dots. It is even conceivable that he is (4) unable 
to relate a printed word by a directional line to the corresponding number of 
dots (although this is least likely). 

This means that the Pre-tests are not diagnostic in any precise sense, 
although probably that is what they should be. Accordingly, the diagnosis- 
prescription process is fairly loose, in this system. Increasing this precision 
would not have the aim of making the system less flexible, and such need not 
be the case. Instead, increased precision of diagnosis would have the purpose 
of enhancing the efficiency of the entire learning system. 

What would be required to improve the precision of diagnosis -assignment? 
First, the Pre-tests would need to be redesigned 30 that each item used to 
measure the accomplishment of a particular objective were followed by other 
items which measured the achievement of subordinate skills . Second, since 
these subordinate skills would have to be known, each objective would have to 
be analyzed so as to relate it to relevant subordinate skills, in the manner in- 
dicated by the discussion in the previous section of this report, entitled "Order- 
ing of Objectives". Were both these efforts to be undertaken, it would then 
be possible to have Pre-tests which provided precise diagnostic information, 
since they would make possible the identification of precise skills the pupil 
had or had not yet achieved. In use, such tests would provide the teacher with 
information making possible an equally precise determination of what the pupil 
needed to do to attain the desired proficiency. 

The Learning Rationale for IMS 

As a learning system, IMS has a rationale which appears highly practical, 
and runs somewhat as follows. First, determine in a rathe. r general way 
what the child already knows how to do, and what he doesn't know how to do 
(using Placement Tests), within the several content areas of elementary mathe- 
matics. Next, determine which objectives he can meet within each area, at 
the level indicated by the previous placement test, this time using Pre-tests. 
Following this, make assignments of skill activities, accompanied by other 
class Activities, which will enable him to acquire the skills (objectr es) 
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i Post-tostH. Further assignments are then 



made on the basis of these results. 

I want to point out to you, not In the sense of criticism, that the IMS Is a 
system of only moderate precision. A highly precise learning system would 
proceed as follows. Following the placement test, a pre-test would be given 
which indicated (1) what objectives the child has already attained; (2) which 
he had not yet attained; and (3) which subordinate skills (objectives) related 
to those identified in (2) were still missing. Having this information, the 
teacher (or other person) would identify for the child the precise skills he 
needed to learn. Having learned these, the child would then be able to achieve 
the particular objectives previously found missing. According to theory, he 
would do this more or less immediately, and without a great deal of "practice". 
The essential conditions of learning would be the availability of these missing 
subordinate skills. Thus, the expectation would be that once these missing 
subordinate skills had been identified and mastered, learning of the new skill 
would occur very rapidly. This would mean greater efficiency for the system 

as a whole. 

Summary and Conclusions 

1. IMS is a learning system for elementary mathematics which has the 
potential for increasing the levels of achievement of children over those ob- 
tainable with a more loosely planned system. In particular, this result may 
be expected because of the procedure of testing--prescription and recycling 
which aims for mastery of the skills of mathematics, and which bases pro- 
gress in assignments on achievement. Because of this procedure, it becomes 
unlikely that children will get "left behind" through being unable to keep up 
with the remainder of a class. 

2. The several excellent features of the system include: (a) well-defined 

objectives; (b) an apparently comprehensive coverage of mathematics skills, 
arranged in sequences which are generally feasible for learning (although 
specific exceptions may exist); (c) well-designed, interesting, and attentLon- 
holding skill exercises appropriate to each objective; (d) a variety of relevant 
class Activities providing considerable flexibility to the teacher; (e) a syste- 
matic set of procedures for placement, pre-testing, assignment, and post- 
testing, designed to make possible student progress in learning based on 
"mastery". 

3. Additional formative evaluation may find it possible to take into account 
the following kinds of possible improvements. These are listed here without 
consideration of their probable costs, but only in view of their desirability. 
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a. IdentLfLcatLon and certain rearrangements of objectives, particu- 
larly across skill areas, based upon analysis of data indicating 
dependence of each skill (objective) upon others. 

b. Increased emphasis in Activities upon application, transfer of 
learning, and problem-solving. This also implies a somewhat 
altered purpose for such Activities, as well as conditions of use, 
from that currently described. 

c. Decrease in the frequency of "testing" versus instruction, with- 
out sacrificing the fundamental assessment purposes of the system. 
For example, it is conceivable that greater reliance could be placed 
upon indications from Check-tests, rather than upon Post-tests. 

A definite recommendation cannot be made here, however, since 
much depends on information derived from usage of the tests in 
an operating situation. 

4. Increased precision as a learning system, would be attained by redesign- 
ing Pre-tests so that they would provide precise diagnostic information. Such 
a change would, however, require a substantial effort, since it would need to 
begin with an analysis of each objective into its component subordinate skills. 

In use, such a system would be expected to decrease the need for "practice" 
of skill exercises, and thus result in greater efficiency of learning. However, 
the practical outcomes of such a system have not been verified; accordingly, a 
recommendation for IMS to undertake it does not seem appropriate. Obviously, 
though, someone should. 
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Numeration 
P 1. 

P 3. 

p 25. 

p 31. 

TA-3 

Numeration 
p 2. 

pp 4-8. 



Numeration 
PP 5-8 

p 12-13 

Numeration 

pp 1-6 



Dr. Joseph M. Scandura 
University of Pennsylvania 
Graduate School of Education 
3700 Walnut Street 
Philadelphia , Pennsylvania 



Specific Suggestions on I.M.S. 



I, Skill 1 

Put another man at the bottom. ' 

Put the 2nd diver closer to the bottom. (He is now In 
the middle, as measured by the ladder) 

The dogs are mirror images, and hence not quite the same. 
Turn one of the dogs around. 

Delete reference, on Contents and Procedure page, to "end 
of the road." The end of the road Is over the horizon. 
Suggest "Circle the car farther from the boy. Mark "X" on 
the car which Is nearer the boy." 

This page does not tell me what I (as a teacher) am 
supposed to do. 



I, Skill 2 

Delete the lines from pole to fish; the child Is to draw 
the lines. 

Almost all of these examples line-up the sets to be matched 
More of them could be spread around a little, e.g. 




I, Skill 3 

Put capital"M"on "More" and capital "L" on "Less". 
(Small letter 1 looks too much like number 1.) 

Use capital M and L. 

I, Skill 4 

There is no need to have all of the pictures In each 
example identical. There could be some variety. 



Numeration 
PP 1-7. 



pp 1-8. 



I, Skill 6 

Contents and Procedure page is not clear about what is 
to be done. Child should put an X on each object in a 
set as he counts. 

See objection to Numeration I, Skill 4. 



Numeration I, Skill 7 

TA-1 The meaning of number sentences must be explained to the 
child; they have not been Introduced before. 



Numeration I, Skill 11 

pp 13-14. It Is not dear, from Contents and Procedure page, what 
Is to be done here. 



Addition I, Skill 3 
pp 8,9. The notation 



G9 — 



etc. 



Is unclear (although It may be clear to the* children), 
p 10. The notation is wrong. In Example 1, NiO^® 1, NfM *1, 
NiOAS » 2. You mean the number of circles, number of 
triangles* etc. Replace N$oS by N^Osi , N}A} by N(A's) etc. 



Addition I, Skill 4 



For clarity, 

pi. 1. 

2 . 

p 2. 1. 

2 . 

p 3. 1. 

2 . 

p 4. 1. 

2 . 

p 6 . 1 . 



the objects being counted should be given. 
How many apples? 

How many birds? 

How many children? 

How many pencils? (Delete the tables) 

How many animals? (Delete the children) 
How many trees? (Delete the squirrel) 
How many teeth? 

How many birds? 

OK 



2. How many men were there then? 

The objective states that the child should use manipulative devices. 
This is not necessary, although It Is possible. Change objective to 
"Solves one-step story problems (sums to 10)." 



Subtraction I, Skill 1 

pp 1-11. Drawing a 

line through the middle of a set does not make It any 
smaller. Sugges.t circling the extra elements of one set 
to make the equivalent. (As in Subtraction I, Skill 2.) 



Subtraction I , Skill 2 

Much of this material (pp. 2, 10-13) requires the reverse 
0 |f the stated objective. The Contents and Procedure page 
Includes "and vice versa" in the statement of the objective. 
This should be added to the booklet statement of the objective. 
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Subtraction I, Skill 3 

pp 14,15. The notation here implies that a set equals a number 
(which of course it does not). Put "N" before each 



set. e.g, 



N 







Subtraction 

pp 6-12. 



P 15. 



I, Skill 4 

The word "less" here refers to an operation. But on 

pp 1-5 the word "less" referred to a relation. To 

avoid possible confusion, suggest replacing "less" on pp 6-12 



by the word "minus". 

The symbol "C)“>1" in the heading has no meaning, 
be deleted. 



It could 



Subtraction I, Skill 5 

Manipulation is not essential here (although it is 
allowed). Restate objective as "Solves on’e-step 
story problems (numbers 1-10)." 
pp 1-6. These pages involve writing a story to describe a 
picture. They do not get at the stated objective. 

Story i is the only one here that asks "How many more?" 
(The others all ask "Ho» t many were left?") Some of 
the other stories could be rewritten to ask "How many 
more?" 

Stories 7 and 11 are the only ones that ask "How many 
more?" 



TA-2 



TA-3 



Multiplication I, Skill 1 ■ “ ' ■ 

p 10. Suggest coloring the noses red or deleting the word "red", 

p 11. As described in the Contents and Procedures page, 

only one child will have the opportunity to discover; 
the others will learn from the explanation. 



p 20. 



The airplanes could be 1 set of 3 or 3 sets of 1; 
point this out in the Contents and Procedures page. 



Multiplication I, Skill 3 

p 5. Example 3. Flyers do not have wings. You could use birds 

instead . 

Division I, Skill 1 

Add (no remainders) to the statement of the objective. 

A. It is not clear how the chart is to be filled in when 

there are remainders . Give an explanation or use only 
examples with no remainders. 
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Fr actions I, Skill 1 



p 9. 



It is not clear whether the cup is cut in half or not. 

In terms of volume it is; in terms of handles it is not. 
Use a glass instead 



& Q 



? 13. 
p 13. 



The headings imply that one* 3 4, one-half = 2, etc. 
Change the "one" heading to "whole", or "all". 

3rd Example. 1/2 of 5 ^ 3. An extra whistle is needed 
In the left column. 



Mixed Operations I, Skill 3 

Change objective to "Solves simple story problems 
Involving addition or subtraction (whole numbers to 10)" 
since, manipulation is not essential. 



Time I, Skill 1 

p 3. Example 2 seems to have nothing to do with the stated objective. 



Measurement I, Skill 1 • 

p 21. The correct heading is "Nearest and Farthest" since 

more than 2 things are compared, 
p 22. Make heading "X Nearer to-v* ". 

p 23. Make heading "X Farther from ". 

TA-3, TA-4. These involve nearest and farthest. 



Measurement I, Skill 2 

TA-1, TA-2. Instructions are needed about what to do when there 
are remainders. 



Numeration II, Skill 2 



p 8. 



p 11. 



Order must be specified. Make titles 
X the first. 

X the third from the left. 

X the fifth from the left. 

X the seventh from the left.. 

(Children cannot be expected to know the usual left to 
right order . ) 

Here first is on the right , which reverses our usual 
order (see comment on p 8.). '• 



Numeration II, Skill 3 

p 5. Suggest grouping these by 10's instead of by 5*s, to show what 

is going on. 

instead of . 



Numeration II, Skill 4 

All of the activities here involve a sequence of number 
lines, rather than a number line. Change objective to 
"Writes the whole numerals from 0 to 25 at the appropriate 
points on a given number line or a given sequence of partial 
number lines." 
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Numeration II, Skill 5 

The expression "Structured groups" implies something 
in abstract algebra. "Structured group" here means 
"set with elements grouped into 10's". Change objective 
to read "States, selects, and writes cardinality of 
sets whose elements are grouped by 10's (to 25 elements)." 
PP 10,11,12,14. 

The correct heading is "How many white rods?" (If you 
allow longer rods, the answer is 1, 2 or 3 in each case, 
which is not what it means.) 

Numeration II, Skill 7 

p 11. is not listed on Contents and Procedures page. 



Numeration II, Skill 8 

P 1 • In order to demonstrate that nuroerosity, not size, is 

inte rest, you could use different sized sets, e.g, 
XX 




p 12, number 7. Replace 27 by 17. (Children are not assumed to 

have mastered 27) 

Addition II, Skill 2 

Recognition of the symbol N for cardinal number is 
implicitly assumed; it is not a stated objective and 
is not tested. 

p 14. We take unions of sets and add cardinal numbers. A 

letter N before each set should be added. - 

C ’ 8 ’ N JS *\+ n£?®k} - N £2 X?) 

3 + 5 «* 8 



Addition II, Skill 3 

pp 6,7,8. Make the number of spaces match the number of letters 
in the word. 



Addition II, Skill 4 

pp 5,6. p 5. cannot be used to help with p 6. since they are 
back to back. 

Addition II, Skill 5 

Delete "Uses manipulative devices to" from objective. 

(they are not required) 

Addition II, Skill 6 

p i. "Yes" can never be a correct answer to the second problem. 

P 7. Kangaroos do not live in the jungle. Suggest you change 

them to lions. ” |: : 
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Subraction II, Skill 1 

P Problem 3. Move sets so they are over the blanks, not over 

the equal sign. 

PP 3-6. These don't involve the stated objective at all. They 

are simply addition problems. To get at the objective you could 
give problems like 



I 






p 10. 



Should be changed as pp 3-6 are changed. 



Subraction II, Skill 2 

There are no examples requiring use of both symbols 
an d "■" to make true statements. Some should be 
added. 

Subtraction II, Skill 4 

PP 9,10. All examples on pp 1-8 are written horizontally, but 

examples onv^-up pages are written vertically. Suggest writing 
some examples vertically on pp 1-8, and some horizontally 
on pp 9-10. 

Multiplication II, Skill 2 

In the objective, replace "pictured" by "given". (As 
worded, objective implies that the child will form the 
array, which is not the case here.) 

The assumption is made that the child knows the "values" 
of different colored rods. This is never stated as an 
objective. _ 

Multiplication II, Skill 6 
pp 4-7, 9-13, 15. 

It would help if distinct shirts, vans, pictures, etc. 
were labeled , perhaps With letters , so the child would 
know they were distinct. 

Division II, Skill 1 

pp 3-12. Suggest replacing "Left" by "Left over". (Left is the 
opposite of right.) 

Fractions II, Skill 1 • 

P 1* Suggest replacing heading "One" by "Whole". 

PP 4,5. Suggest replacing heading "One" by "all". (It appears as 

written, that one * 6, one * 4, etc.) 
pp 8-12. Suggest changing notation, as follows: 

v' One-half 



• i°o\ 
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Fractions II, Skill 2 

p 12, Example 2. Add another dot. 

p 16. To clarify notation, suggest changing example 2 to read 

1/2 of (O's +£>’s) 

(Oand ^ are used as icons here, not as pronumerals.) 

Mixed Operations II, Skill 1 

pp 7_9 a standard convention when using pronumerals is that in 

a single equation, the same number must go into a given 
shape, no matter how many times that shape is repeated. 

If two different numbers are allowed, 2 different shapes 
must be used. Therefore, for all equations with 3 numbers 
to be filled in use 3 symbols, e.g.O 

r *. 

Mixed Operations II, Skill 2 

p 7, Example 2. Carrier pigeons do not carry children! 

Money II, Skill 3 

pp 3,4. Indicate to teachers that there is more than one correct 

answer, e.g. , 8<? buys 2 balls (2$ left over) or 4 pencils, 

or 2 balls and 1 pencil, or 1 pencil and 1 apple (1$ left over). 

Items could be chosen to make prices more realistic. 

Time II, Skill 2 , 

pp 5-6 The objective Implies the ability to fill in all numerals. 

One clock face should be given with none of the values 
filled in. 

Measurement II, Skill 1 

p 1, Example 3. Neither line is shorter. Suggest putting the whole 
sentence on 1 line so the reader will know he is to fill 
blanks. (It looks like he is to compare the length of 
line segment A and line segment a.) 

Numeration III, Skill 2 

pp 3,4. A starting point should be given. For example, the hidden 
number on p 4. can be 48 or 24 , depending on where one 
; start 8 . 

Numeration III, Skill 5 

See objection to Numeration II, Skill 5, Change objective 
to "States , selects , and writes cardinality of. sc&ts whose 
r elements are grouped by 10* s (to 100 elements)." 
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Numeration III, Skill 8 

p 1, Change lines so that they do not seem to form an inequality 

sign. (See Subtraction IX, Skill 1.) 



Addition III, Skill 5 

p 5. A capital N is needed in front of each set. 

cardinal numbers, we take unions of sets;. 
Skill 2.) 



We add 

(See Addition II, 



Addition III, Skill 3 . 

A. In number sentences like£j+ * *— 4 » use distinct 

pronumerals, e.g. C3 + * ^ , unless the numbers 

are supposed to be the same. 



Division III, Skill 1 



P 


9 __ 

Very confusing, 
subtraction and 
Suggest instead 


Some arrows (horizontal ones) represent 
some (slanting ones) represent equality. 




( 3 ) 


take away 2 is 


© 


8 -©« 




© 


take away 2 is 


© 


6 



6 



A 



etc. 



Division III, Skill 2 

pp 8,9. It really is not necessary to use rods here. All you 
have to do is count. 

p 12. To fit the accepted convention on pronumerals suggest changing c 

5. O x ^ * 9 

6. & x CD - 16 
9. O 

xft 

15 

Fractions III, Skill 2 

p 12, Example 5. This is not, divided into halves, but one can shade 

one half. Change this to halves or to thirds. 

Example 9. Same as Example 5. Change this to thirds or to 
fourths. 



Time III, Skill 3 

All of these pages may cause later confusion. 

"36 marks to the 4^ " means "36 minutes after the hour 
and' ”24 marks after the 4^ " means "24 minutes before, the 

hour." Suggest change all of these to replace ma 

after 4> " with " marks to 12." This will avoid 

confusion with the word"a£ter". 
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Measurement III, Skill 1 

p 5. Unclear. What is number 6? Docs number 8 refer to the 

diameter of the reel or the length of the tape? 

Measurement III, Skill 5 

p 3. Unclear. Does number 3 refer to one paper clip or the 

whole box? Does number 8 refer to one staple or the whole 
box? Does number 9 refer to one tissue or the whole box? 

Numeration IV, Skill 4 

Change the objective to make clear what a "structured 
group" is. (See Numeration II, Skill 5 and Numeration Hit 
Skill 5.) 

Numeration IV, Skill 5 

p 5. The pattern in numbers 3, 4 and 5 Is not clear; how are 

children supposed to figure it out? 

p 6. Use blanks instead of squares to avoid the pronumeral 

problem. 

Numeration IV, Skill 8 

p 1. Replace "one before five means 1 less than 5" by "I 

before V means 1 less than 5"; replace "one before ten 
means 1 less than 10" by "I before X means 1 less than 
10 ". 

Addition IV, Skill 1 

p 6. Problem 2 and 5 are confusing; suggest deleting them or replacing 

them by problems like number 6. 

This page could Include some problems using zero. 

Addition IV, Skill 5 

Change objective to "Solves column addition problems 
with three or more single digit addends (whole number 
sums to 25)." 

Subtraction IV, Skill 1 

pp 3-5. Label jan intermediate point on the number line, e.g. 

9 or 10. 

p 8. To make the subtraction triangle here like the ones on 

pp 9-10, it should have 











Subtraction IV, Skill 2 

p 2. Make shading clearer. 

p 3. Up to the point, every block has represented 1. Now 

some of them represent 10. To make this clear, suggest writing 
the ones in the left column with T's e.g.jjJ 
Then problem 1 becomes 



4 3 

- 1 2 

31 

(Shading seems bad as it stands , because only shaded blocks 
were subtracted before.) 



T 




T 




X. 


S3 



Multiplication IV, Skill 2 

p 4. Very confusing. Suggest making the number line horizontal 

and having it go from 1 to 40. 

TA-3. 2 lines up from bottom. Replace "3 x 4" by "4 x 3". 

Multiplication IV, Skill 6 

The objective implies that the child can do' pages like 
page 7 with all of the numbers left blank. There should 
be some problems like this. 



Multiplication IV, Skill 7 



p 3. 
p 4. 



p 5,8. 



Give one more answer in each column to make the pattern 
clear. 

You could delete the shading. The fact that 2x2 • 2..x 2 is just 
one more example of the commutative law. It is important 
that the child know that in the number sentence Zj+D b D+/j, 
the numbers may be the same . 

Delete the shading (except where the product is too large). 



Multiplication IV, Skill 9 

Basic assumptions 
p 1 no. 1. Each 
p 2 no. 1. Each 
p 3 no. 3. Each 
p 7 no . 2 . Each 



should be stated as follows: 
coach needs 4 horses, 
load has 5 poles. 

8 tool has 3 legs, 
sloth has 4 legs . 



Fractions IV, Skill 5 

In the objective, replace "using pictures" by "given 
pictures." If the child is to construct his own pictures, 
as implied by the present wording, then extra pages must 
be added requiring this. 



Money IV, Skill 1 

p 1. Delete 0-*in each case; it is confusing. Just leave 

a blank. 
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Measurement V, 
P 7» 



Skill 1 

An empty bushel basket should be Included here as measuring 

Instrument for use with the apples • Delete M 1 peck 

1 bushel" . 



1 Numeration VI, 
p 17,18. 



Skill 7 

Heading on last column is ambiguous. Suggest changing to 
"Neither prime nor composite" or just "Neither". 



Multiplication 



^Multiplication 



VI, Skill 1 

A review page is needed at the beginning to do problems 

such as 3 x 10 * , 4 x.50 ■ , 5 x 70 * (as in 

Multiplication V, Skill 6). 

VI, Skill 2 

Between pages 9 and 10 there should be a review page of 
problems to be done in their head , such as 
3x4 + 2 
6x8 + 3 
7x5 + 4 etc. 

This is a prerequisite skill for pp 12-16. 



Multiplication VI, Skill 6 

The short cut procedure given on pp 13-14 was already 
given in Numeration VI, Skill 8. It could be used 
from the start on this skill. 

Multiplication VI, Skill 8 . 

p 12. In problem 7, there is no higher count on the 5 s. 

f In problem 8, there is no higher count on the 7's. 

t In problem 11, an exponent is needed on the first 5. 

t Division VI, Skill 1 

Heeds a review page with problems like 3 x 40 * * 

| 7 x 30 - . 

^Division VI, Skill 2 ^ . 

f pp 8,9. A vorked-out problem where you subtract 200 fours would 

be helpful. Suggest changing problem 1 on p 9. to 

- 227 

100 fours 



100 fours 
10 fours 
10 fours 






4 



1908 

-400 






i 



V 



7 fours 
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Division VI, Skill 2 pp 18,19,21. 

Skill 3 pp 3,5,6. 

Skill 4 pp 3,5,6. 

More space is needed on these pages because the child 
must use the distributive law to do the divisions. 

Division VI, Skill 4 

p 5. Problem 2. Lost line should be 

7fl73* - (instead of 5(173 - ). 

Fractions VI, Skill 2 

This entire section should be rewritten. It makes 
extensive use of the unjustified fact that 

a c axe 

b x d " b x d * 

It also uses the unjustified assumption that 
a . c a jt c 
b 7 c b 7 c * 

Fractions VI, Skill 3 

pp 8,9. Delete. These use the unjustified fact that 

a ' c ■ a . c . 

b * d b d 

p 12. Delete last half. 

Mixed Operations VI 

Suggest replacing the word "variable” by the word unknown In 
the statement of the objective. 

p 10. The 10 litre GT is especially neat. But how many people 

will get the joke? 

Mixed Operations VI, Skill 4 

The left to right order is a correct convention on 
addition and subtraction. But it is not an accepted 
convention on multiplication -and division. That is, 

8 v 4 x 2 is ambiguous without parentheses, 
pp 4-12. Should be rewritten to include the left to right 

convention on addition and subtraction, but it should 
be pointed out that there is no convention on multiplication 
and division. 

Time VI, Skill 3 

pp 5-10. These problems suffer from the same ambiguity as those 
in Time V, Skill 6, p 6. Use of the phrase, "days 
(or weeks) later", may clear up some of the ambiguity 
(e.g. Bill got to Germany 2 weeks later, Eddie finished 
the tree house 3 weeks later). 
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Measurement VI, Skill 1 

p i. You do not use inches of wood. You use square Inches. 

Rewrite the problem. 

Measurement VI, Skill 2 

p 2. Number 1. Preserves are also sold by weight. Since 

either answer is correct, suggest deleting the problem. 

Numeration VII, Skill 1 

p 7. A VW weighs about 2,000 pounds, not 4,000 pounds. 
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GENKRAL REPORT ON I.M.S 



PART A. ADDITIONAL GENERAL COMMENTS 



The I.M.S. materials seem to provide a useful set of materials for 
teaching the usual elementary school arithmetic skills. The objectives 
ate specific enough to be useful, without being overly specific. With 
the exception of the specific changes suggested in the first report, they 
are clear, comprehensive, and precise. 

The teaching materials provide for achievement of the stated objectives. 
They should be suitable for a wide spectrum of schools and children. The 
materials should enable most children to achieve the stated objectives. 
However, it should be recognized that some children will not meet the 
stated objectives after using these materials only. Specific suggestions 
should be made to the teacher for additional work on each major type of 
objective for children who do not reach criterion. Better yet, supplementary 
materials might be prepared (see Part B of this report). 

The following are some general reservations about the I.M.S. materials. 

1. Arrows are used indiscriminately with no Immediately apparent 
thought or planning. In mathematics, arrows are generally used to 
represent functions (or mapping or correspondences) and vectors. In these 
materials they are used in many ways, with no apparent pattern (see 
Numeration I, Skill 10; Addition I, Skills 2 and 3; Subtraction I, Skills 
3 and 5; Fractions II, Skill 1; Division III, Skill 1 (on page 4, arrows 

] have two different meanings on the same page) ; Money IV, Skill 1) . 

! 

All uses of arrows should be checked, to see if some order can be 
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created in their use. 

2. The general convention of pronumerals is not followed. That is, 
in the number sentence □ + A - 8. one is allowed to substitute distinct 
(or equal) numbers for P and A. But when a pronumeral is repeated in 
a number sentence (e.g. D +□- 8) you must substitute the same number 
in each case. The l.M.S. materials violate this general convention. 

An attempt was made to list all places where this occurred on specific 
pages. 

3. Children may need help with some of these materials. The 
individual pages, in many instances, do not make clear what needs to be 
done. Even the Content and Procedures pages are not specific enough in 
some cases • 

Some specific places where this occurs are: 

Money 1, Skill 2, p 6 
Measurement 1, Skill 1, pp 16,17 
Measurement II, Skill 1 (all pages) 

Measurement II, Skill A, pp 9,10 
Multiplication V, Skill 5, p 4 

A. The meaning of division is reversed part way through the materials, 
but this is never pointed out. In Division III, Skill 1, it is stated 
that in a f b - c, a - the total number of elements, b represents the 
number of subsets, and c represents the number of Elements in each equal 

subset . 

Later this is reversed, so that b represents the number of elesients 
in each subset and c represents the number of subsets. The relationship 
between the 2 kinds of problems is never made clear. (It is stated that 
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repeated subtraction solves the second situation.) The fact that the 

same division problem can be used to solve both kinds of example should 
be made dear* 




5. The fact that (a x b) y c - a x (b f c) Is used In Division VI, 
Skill 1 but It Is never expllcltely justified. 

6. Some pages in several sections require abilities beyond the ‘ 
stated objective. However, in general, the. additional ability is not 
required on the check-up pages. 

Some specific places where this occurs are. 

Mixed Operations I, Skill 3, pp 1-4 
Time I, Skill 1, p 3 
Numeration II, Skill 8, pp 17,18 
Subtraction II, Skill 5, pp 1-5 
Numeration IV, Skill 5, p 1 
Measurement V, Skill 2, p 6 
Measurement V, Skill 3, pp 7,8 

1 • In Numeration III, Skill 11, the general structural properties 
of Roman numerals are not explained, nor is the general subtraction principle 
explained. Everything is strictly rote. Even p. 12 seems to require 
translation to Arabic numerals and then translation of the sum back to 
Roman numerals. Some explanation of the general procedures for writing 
Roman numerals would be desirable. 

The same applies to Numeration IV, Skill 8. 

8* Many number lines are represented as sequences of shorter number 
lines. Whenever possible, it might be better if the page was turned 
sideways, the scale made smaller, and the entire Interval under consideration 
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put on a single line. 

9. In many instances (e.g. D * A ■ 20) , there are several correct 



1 answers . 


This should be pointed out to the teacher. 

i 
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PART B. SUGGESTED NEW DIRECTIONS 

Although the I.M.S. materials cover the stated content objectives 
quite adequately (subject, to the reservations stated in Part A.) there 
are three basic needs which are not met directly. Several members of 
the Mathematics Education Research Group (MERG) are engaged in development 
projects aimed at these needs. They merit attention by R.E.L.C.V. as a possible 
direction for further development in conjunction with MERG. 

The purpose of the first is to devise efficient , self— instructional 
materials to teach and/or provide review in the basic arithmetical skills 
of addition, subtraction, multiplication, and division. It ia highly likely 
that some children will not master these skills after using the I.M.S. 
materials, and efficient back-up materials are needed. The second project 
Is aimed at developing a systematic way of teaching children to thin): 
critically while reading (i.e., to read critically), and is based on a 
behavioral analysis of the process* in terms of logical reasoning. To these, 

I would add the critical need to train teachers how to systematically 
provide their students with opportunities to Increase their ability to 
learn new material of a technical sort, to communicate with precision, 
and to reason logically. While not normally considered to be part of 
mathematics, these basic processing abilities are critical not only in 
mathematics Itself, where they obtain perhaps their clearest expression, 

but also in the everyday world of reality. 

It is easy to justify inclusion of these three projects in further 
development aimed at an idealised mathematics curriculum. Children in 
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thc Philadelphia schools, for example, particularly those in West and 
North Philadelphia, average several years behind the national norms 
in arithmetic. Many of these children could be expected to learn the 
arithmetic skills from the I.M.S. materials. But many others would 
undoubtedly fail to master them. Working individually with these children 
could severely limit the time the teacher has available for dealing with many 
of the equally, if not more. Important aspects of mathematics education; 
meaning, processing skills (e.g., reasoning), and relationships to other 
subject areas and to reality are some of the more obvious. Hopefully, 
the Inclusion of back-up self -instructional aids for teaching the basic 
arithmetical skills will relieve the teacher of part of this responsibility, 
thereby providing more time for emphasizing meaning, processing skills 
and the rest. It cannot be taken for granted that teachers will automatically 
do this, however, and the other two projects are designed to help Insure 
that the change does occur. 

One of the most critical of these needs is for the student to see the 
Importance of precise logical thinking ■ in areas outside of mathematics 
as well as within the subject itself. The ability to read critically has 
to be one of the more Important areas in which this skill is needed. This 
is an area where development la definitely needed, because in spite of the 
concern many have expressed about critical reading, very few materials 
exist for teaching it. Educational products which are designed to deal 
with the problem in a systematic way are nonexistent. The difficulty is 
that there has been no adequate conceptual base for dealing with the problem. 
MERG feels that it has such a base and Is presently trying to demonstrate 
its value. 
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It is also hoped that by providing teachers with self-administered 
instructional aids to use with their pupils (e.g. » the arithmetic skills 
materials), they will have more time and be more willing to ehgngo in n 
period of inscrvice education and/or classroom practice designed to improve 
their ability to teach basic processing skills. In view of their great 
importance, one could hardly do otherwise than to make provision at the 
earliest possible time for improving such skills in any educational program 
designed for educating the disadvantaged child. 

In the remainder of this section, specific projecto for meeting 
these three needs are described, together with descriptions of what has 
been done so far. 

1. Arithmetical Skills Project 

The arithmetic skills project has been sponsored by the Mathematics 
Education Research Group (MERG) on a limited basis for approximately one 
year. The work has been done under my direction by Jeannine Gramick, 

Debra Whitley, and John Durnin. At the present time, the major part of 
| the engineering described below, as well ss some of the development and 

I evaluation of prototypes, has been accomplished. Large scale development 

§ 

I and revisions remain for the future. 

1 Recent basic, theoretical advances here have provided us with a basis 

1 for ordering competencies (i • e . , rules , algorithms , or paths thereof) 

I according to difficulty. This type of ordering makes it possible to 

1 increase the efficiency of the assessment procedure and to systematically 
sequence the instruction in teaching each of the arithmetical skills. 

The first step in engineering the project was to refine the 
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(technological) goal - that Is to spool ty those aspects ot the total 
project which night be engineered by building on available conceptualizations, 
in the present case, our goals were to Identify: (a) the nost efficient 
procedure possible for performing each kind of computation, <b) the best 
way to sequence Instruction on the parts (i.e. , the paths) of this 
procedure, and (c) an efficient method for assessing mastery at each 

stage. 

The next step was either to adopt an existing engineering plan, or 
to devise a new one, for achieving the above goals, (a) In order to 
find the most efficient procedures possible, which were compatible with 
the usual computational algorithms, we proceeded as follows. For each 
arithmetic operation, flow charts were prepared for the common algorithms 
in order to specify precisely what was Involved at each step. Programming 
techniques were then used to combine the alternative algorithms, to 
eliminate redundancies, and otherwise come up with the best possible 
algorithm. This typically Involved minimising the number of path, involved, 
but such considerations as difficulty of the paths, and prerequisite 
abilities which could safely be assumed of the pupils, were also involved. 
The procedure used for subtraction can be represented as follow.. 



Insert Figure A here 
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(1) Start at 
rightmost column 
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Figurc A 



(2) Subtract the bottom 
number from top number 



4a Put a 1 In front 
of the top number 

T" ' — 



4b Subtract the bottom number 

from new top number using 

subtraction fActs^lO 
~ — 



4c Go to the next column 
and add 1 to the bottom 
number 



Directed Craph 



START 

_(1> 



Paths 



STOP 



«) 






♦ill_OC sl-U.L 

Uz) MJ 



e-XU.>JC2JL^ ^.f 



m 




(3) 

(A) 



ERJC 

k 



184 



(3)Go to the 
next column 



2 . 



no 




) — » 


STOP 






Instances 
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- 241 
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-875 
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(h) The directed graph at the bottom of the figure in a form of 
repreoentntion we have developed, which is useful both ip pinpointing 
equivalence classes of tasks for testing and in arranging paths according to 
difficulty. In fact, all that is necessary in arranging (sequencing) paths 
according to difficulty is to choose the nost direct path (e.g. , Path 1) 
first, then the next nost direct paths (e.g.. Paths 2 and 3)*— there nay 
be aore than one, and so on. The partial ordering laposed in this nanner 
on the above subtraction algorithm My be represented by the following lattice. 



It is this ordering then that provides the bnsls for sequencing the 
instruction. 

(c) The ordering provides the basis for sequencing the testing. 
Our assessment procedure involves selecting Just single test instances 
from each equivalence class of tasks (where each path corresponds to 
a distinct equivalence class). The ordering of paths according to 
difficulty, increases the efficiency further by ma king it possible to 
introduce a conditional testing procedure. The basic rationale is that 
if a pupil succeeds on tasks corresponding to more difficult paths in 
a hierarchy, then there is no need to test him on less difficult ones. 
Vhere the more difficult task is failed, of course, additional testing 



Path 4 




a. 



Path 1 



will be necessary. 

In the above discussion, we have used the algorithm for subtraction 
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lo J litis irate tin' eng I ncering plan for nolil*»vl tip, l lie iiulic.itcd goals. 

This process tins also been carricit out for the addition, subtraction, 
multiplication; and division algorithms. * Thus, at the present time, 
we have completed most of the engineering for computation with the whole 
numbers and could begin work with fractions and decimals if resources 
were available. 

The actual developmen t of usable products is at the prototype 
stage. The plan used to date involves the eventual use of two- or four- 
track cassette tape recorders, together with student workbooks. The idea 
is to provide the child with a self-instructional mode which he can use 
to work at his own pace, and to minimise the amount of effort required 
of the teacher so that she will be free to devote herself to the explanation 
and exploration of important underlying concepts. As a first step in 
this direction, we have been working on preliminary versions of the f 

4 

i 

workbooks and scripts for the tapes. Due to cost factors, the latter 
have been tried only with single-track tapes. 

Tentative plans call for building in motivation through a variety 
of devices. One such device that has already proved useful involves 
putting teaching and review in a game context where possible. Another 
way to increase motivation would involve the use of four-track tape 
recorders. (Such recorders will also be necessary if we are to capitalise 
fully on the engineering that has already been done. Thus, in addition 
to switching among different cassette tapes according to His needs, 
the child would be able to switch from track to track within any 
given tape.) Our hopes are that providing the child with a choice 
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of tliickfj (l.e., a variety of review modes — some, involving games) 
will tend to increase his span of attention, and inject a bit of the 
unexpected (i.e. 9 "What kind of review am I going to get?") into an 
otherwise repetitive, but important, task. In addition to providing 
variety, we also plan to reward success more directly by including a 
few minutes' worth of music, Jokes, and/or humorous commercials and 

dialogue ou the "success" channels. 

Although the development plan has appeared promising In preliminary 
tryouts, the skills project dearly points to the need for further basic 
work in the areas of motivation and memory . Hopefully , such work will lead 
to the development of systematic engineering techniijucs which will make 
it possible to deal with these aspects of development In a more effective 
and efficient way. 

2. Critical Reading Project 

The immediate goal of this project is the development of aelf- 
instructional materials for testing and teaching critical reading skills 
in urban schools. Many children who have learned to read in a more or 
less technical sense cannot read critically. They can translate written 
statements into sounds and know what these statements mean, but ideas come 
through as fragments which are, for the most part, unrelated to the more 
comprehensive whole. Ultimately, we hope to expand the scope of the project 
to deal with reasoning more generally, and at younger age levels . (Some 
suggestions are sketched below for moving in this direction.) 

In helping to meet these needs we have been able to draw on two basic 
ideas (conceptualisations) and one technology. A behavioral analysis of 
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thc process by which meanings are related reveals the involvement of 
what arc called semantic inference rules. Semantic inference rules 
parallel the syntactic versions characteristic of symbolic logic, and 
operate on meanings. 

The second conceptualisation is at base a taxonomy for classifying 
meaningful reading contexts according to the generality of the semantic 
inference rule required to reason effectively in that context. Civen 
any particular (syntactic) Inference rule, for example **A implies B,” 

A; ' therefore, "B," the generality of the corresponding (semantic) 

Inference rule needed would depend, for example, on the amount of Irrelevant 
information contained in the reading context. The more redundant 
information included, the more general the(semantlc) rule required. In 

» * ive essentially independent dimensions, ordered according to difficulty 
(and hence, according to degree of generality required of the underlying 

semantic Inference rule), were identified along which reading contexts 
can vary. 

In addition to these conceptualisations, qua conceptualisations, a 
potentially useful engineering technology has already been developed. 

This involves a technique for assessing behavior potential on classes of 
tasks which vary along a set of independent dimensions . The technique 
is described in its original fora in Scandura (1968) building on the 
experimental results of Scandura and Durnln (1968). 

These conceptualisations provided a basis for the systematic 
attainment of the following aspects (l.e., the technological goal) of 
the critical reading project. The first goal was to specify the kind of 
behaviors involved in the use of each semantic Inference rule, e.g., the 
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alii lily to driccl valid inference* , coni tadlcloi y statement*, and Invalid 
inferences, nil in the context of wrillcn materials. The second was to 
specify the dimensions, and the levels along these dimensions, over 
which the stimulus materials for each inference rule any vary. The third 
■ajor goal was to determine the most efficient way to sequence the testing 
and instruction so as to determine In exactly what reading contexts each 

child can use each logical rule, and to expand these contexts as much as 
possible. 

The engineering plan used involved first selecting the most common 
logical inference rules and specifying precisely the various dimensions, 
and levels along these dimensions, over which actual stimulus contexts 
(i.e., reading passages requiring use of these rules) may be realised. 

Next, we had to actually construct, for each inference rule, reading passages 
with specified levels along each of the dimensions, We then had to refine 
the testing procedure given in Scandura (1968) and to extend it so as to 
sequence the instruction from level to level. Because it is not yet feasible 
to attempt a fully algorithmic analysis of semantic Inference rules, however, 
some form of inductive method (i.e., learning by example) must be used 
within any given level. A direct expository approach requires precise 
specification of underlying rules. 

In order to actually develop a usable product, it was necessary to 
incorporate the engineered materials into a more general plan which took 
other factors into account, motivation being perhaps the most crucial. WO 
attempted to build motivation into the materials by making the "stories" 
short and as interesting as possible. Short, frequent subtests were also 
included, so that each child would not only get adequate reinforcement and 
sufficient practice, but so that he would get extra practice where needed. 
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Aft In «»1 1 drvul u|»im*nl , of courm*, wi* nhio lt.nl lo dt*.i1 with quo?; I Inns 
of forunt . Aflrr full con?: Itlcr.il Ion of lltc ul t ernat Ivon , il wan doc lilt'd 
lo initially limit ourselves to a read Inf, mode, specifically n mode 
designed for children reading at the third and fourth grade levels. Work- 
books and pre-recorded tapes arc now being prepared. Although they are 
primarily designed for self-instruction, these materials could also be 
used in group instruction. Here, the teacher would have the students use 
the workbooks and generally follow the scripts from which the tapes are 
to be made. 

Fven if the initial tryouts of the prototypes are successful, we 
want to keep open the possibility of trying other nodes before undertaking 
large scale development. It may be desirable, for example, to modify the 
materials for use with younger children by introducing a pictorial and/or 
aural mode. In particular, using such modes could make it possible to 
work with nonreaders as well as readers. 

3. Processing Skills Project 

In this final section, we describe a development project aimed at 
meeting the critical need for awareness and successful teaching of the 
more general process abilities, which are critical in performing a wide 
variety of mathematical tasks. As many schools presently exist, this 
need is twofold: first, to Increase the teacher’s awareness of and 
training in these processing skills; and second, to develop instructional 
material for the student which maximises his opportunity to acquire these 
skills. 

The conceptual base for this project Includes the taxonomy of 



190 



proci’N.*; abilities In waUiow.it ica which in described f u ljy | n Chapter 1, 
M.nluraaL i crete J * H i/\v l_or^» 1 JPom i td.- it ioiut. * hr J c f J y these ahillt icr. 



arc: (1) detecting regularities, and the reverse ability of (2) parti- 
cularizing; (3) interpreting mathematical descriptions, and the reverse 



ability of (4) describing mathematical ideas; (5) Baking logical inferences, 
and the reverse ability of (6) axiomatiting. This conceptualisation 

derives from an attempt to define operationally what it means to "think 
nathcmatically , 11 

In addition to this taxonomy, which by itself would constitute a 
fairly limited conceptualisation, we have available a theory of knowledge 
which provides the basis for an "algorithmic analysis". This analysis 
involves systematic identification of the tasks to be learned, formulation 
of these tasks in terms of (observable) behaviors to be elicited, 
formulation of an efficient role for solving each task, elimination of 
redundancies, searching for higher-order rules, and finally eliminating 
| those rules which can be derived by application of higher-order rules. 

| finally, logical analysis of the rules Introduced to generate 

I 

various classes of tasks provides a conceptual base for Gagne's "task 
analysis, which may be used for the identification of prerequisite 
abilities. In describing the engineering phase of the proposed project, 
j we 8ha11 sec ho * these conceptualisations can be systematically brought 
to bear in the attainment of our goals. 

The first step in engineering the plan is to refine the (techno- 
logical) goal — i.e., to identify those elements in the development 
that can be dealt with systematically, based or available conceptualisations. 
Evidently the following aspects of the present project should be susceptible 



*Harpei & Row, 1971. 

O 
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to this kind of engineering: (a) developing Introductory materials 
explaining the taxonomy to pre-nin) elementary school teachers, and 
training the* explicitly in the processing skills; and (h) developing 
specific curriculum materials for use in the schools which systematically 
teach and emphasise the. use of processing skills In performing a variety 
of mathematical tasks.* 

The second phase is to devise systematic techniques for achieving 
the two technological goals above, based on the indicated conceptualisations . 
(a) The processing skills themselves as well as the overall taxonomy, 
constitute the subject matter content of the introductory material 
for sathcsatlcs teachers. Here, the technique would be to apply 
"algorithmic analysis" to the existing descriptive literature, in 
order to develop material which teacher the subject more efficiently. 

(b) To develop the curriculum material for students, we would 
first collect subject matter (from existing school curricula) illustrative 
of the processing skills, and systematically identify the process 
abilities Involved using the t axon omy. Other tasks requiring 
processing skills would be devised from scratch to fill voids in the 
taxonomy. To the extent feasible, the material selected would then be 
systematically sequenced using task and/or algorithmic analysis. 

The third phase is the actual application of these techniques to 
achieve an engineered plan. Some work in this direction has already 
been carried out — in particular, members of the Mathematics Education 
Research Group have completed an algorithmic analysis of Chapter 1, 



*At the present time, we leave open the question of how closely such 
curriculum materials should parallel existing subject flutter curricula 
in mathematics. While a close parallel Slight facilitate the immediate 
applicability of our material in the schools, it could diminish the 
efficiency of teaching the processing skills. 
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In the fourth And final phase, those responsible for the engineering 
would cooperate with those responsible for the development, in identifying 
explicitly those aspects of the development which must be dealt with on 
more intuitive grounds. For example, here we can identify the motivation 
of teachers, motivation of students, decisions regarding media or pre- 
sentation, and those aspects of the sequencing not governed by task 
and/or algorithmic analysis. 

The responsibility of the developers is to com up with and implement 
a practical plan for Incorporating the engineering into a usable product, 
(a) For example, having decided on the potential value of a workbook 
for training teachers in the taxonomy of process abilities, the developers 
must rework the engineered materials into a workbook format so as to 
build in the non-engineered elements of motivation, sequencing, and so 
on.* (b) Likewise, a medium of presentation would be selected for the 
curriculum material, and the developers would have the responsibility of 
synthesising the engineering and the non-engineered aspects such as 
motivation into a final product suitable for the schools. 



*This process is now complete for the workbook accom pany ing Chapter I of 
Mathematics; Concrete Behavioral Foundations. Task analysis was not 
used to engineer the sequencing of the workbook suterial; this was 
inherent in the chapter itself. 
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December 18, 1970 



The next EC Meeting will be held January 15, 1971, in Durham. 
I will advise you of the exact location and agenda the first week 
of January. 

Enclosed is a set of materials contributing to our evaluation 
as follows: 



t 

f. 

t 



r. 




1. The first Teacher Survey - Please get all teachers who are 
or will be using IMS this year to fill out a copy. If you 
are a classroom teacher please out a copy yourself. 

2. Incident Report Forms - Please duplicate additional copies 
of these if you need them or collect incident reports on 
plain paper. If materials are involved, the Incident Re- 
port should identify the page and problem that caused 
trouble. The few we have recived thus far are inadequate 
in that they contain comments, not the incidents that gave 
rise to the comments. There are places on the Survey for 
comments and opinions . 

3. Error List #3 - These are. errors which . you reported that 
were not on any prior list. Please continue to collect 
errors. Incident Reports are the. best way, since finding 
an error is an incident. However, please ask teachers to 
explain and give level, page, and problem number of errors 
carefully. Some we received could not be tracked down. 

4. A Census Report Form - Please fill this one out for the 
period ending January 15, 1971, just as you did for the one 
covering through November 30, 1970. 

If more classes or groups have' moved into IMS in your school 
since November 30, please record them on a separate report 
form. (Two are provided in case you need them.) 






December 18, 1970 
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1 

Please bring all of the above items with you to Durham in Janu- 
ary. There is no need to mail them ahead of time unless this is 
more convenient. 

j 

Thank you for your cooperation and let me know if you require 
any further information. 

Sincerely, 

I Robert B. Frary 

| Research Associate 

RBF:clg 
Enclosures 





reGionaL eDUcanon LaBoraiorY 
For THe caroLinas anD virGinia 



March 8, 1971 



Enclosed are copies of two new survey forms. Let me explain each 
in turn: 

1. EC Questionnaire (one copy only) Please fill this out so that 
we can standardize some of our information about schools. 

2. Teacher Survey #2 This survey is only for teachers now using 
IMS. It clarifies some of the questions raised by the 
earlier survey and seeks some new information. 

Please return the completed forms to me by mail as soon as possible. 

As you may know by now, it was necessary to change the data of 
the principals' meeting to March 19. The next EC meeting will 
take place as planned in April. At that time we will probably 
want to get your ideas on how many of each folder should be stocked 
in your school for future usage. The date will be announced in 
early April (probably April 16 or 23). Do you have a preference? 

Have you had an opportunity to collect any children ;'s favorite 
story lines for use in IMS-2? You may- remember my. mentioning 
this -.at the- last EC meeting*: Any documentation you.eould provide 
would ; be helpful. Perhaps you could get some- imaginative themes 
from- children in your school or a nearby junior* : .high school. 
Mathematical content is not necessary . 

Thank you very much for your help. 



Sincerely, 



Robert B. Frary 
Research Associate 

Enclosures 

RBF/swb 
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REGIONAL EDUCATION LABORATORY 
FOR THE CAROLINAS AND VIRGINIA 

May 21, 1971 



Dear Teacher: 



This is the final IMS survey which should be answered anony- 
mously. A number of questions on earlier surveys are being 
repeated for two reasons: 

1) A number of responders omitted items on earlier 
surveys, thus indicating they had not reached 

a conclusion. On this survey, please do not 
omit items. If you are not sure, mark the choice 
which seems closer to your belief. 

2) In some cases opinions change with time, and we 
want to record any shifts that may have occurred. 

Thank you for your cooperation both in using IMS and in helping 
in our formative evaluation. Because of your assistance, 

IMS should be a much better system as it is used in more and 

more schools . 

Sincerely, 



Robert B. Frary 
Research Associate 



Enclosure 

RBF/swb 



Mutual 




(Chapel Hill and Duke Streets) Durham, North Carolina 




27701 



Plaza 



reGionaL eDUcanon LaBoraTory 
For me caroLinas ano virGinia 



May 21, 1971 



\ 



| Here is the final set of evaluation materials for your school. 

! Please give each teacher using IMS a questionnaire and em- 

r phasize that we would appreciate their marking every item 

! even if they have not completely made up their minds on a 

| question. (This is explained on the questionnaire cover 
| letter.) Of course, you should fill out a copy. Since 
the questionnaires are anonymous, please keep a checklist 
\ so that you can tell when all teachers have returned 

We would like to have 100% returns on this questionnaire. 

i 

!i 

| Census forms are enclosed for the end-of-year results. Please 

I remember to report the levels the students will enter next, 

I not the levels they have completed. In other words, base 
I the final report on anticipated fall, 1971, placement 
I (assuming no loss over the summer) . 

| Please return the completed questionnaires and census data as 
| soon as possible. Final census results will be returned to 
I you shortly. The final report summarizing all evaluation 
I outcomes will be completed late in the summer and multiple 
I copies will be sent to all schools. 

I Finally, thank you for your fine cooperation throughout the 

| year. It is a big job to make real improvements in the way 

children are taught, and you have done more than your 
share. 




o 

ERIC 

I . .. mi r.*i. •*> :< .» -ir 
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n<\sl wishes lor now . 1 hope wo will have on opportunity t 

moot or work toejothor aqain. 



Sincerely, 



Robert B. Frary 
Research Associate 

Enclosures 

RBF/swb 
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Results of the Item Analysis of Posttcsts 



Test 



Num. II 



Mo. of •«* 
Cases to 



112 



Add. II 



Subt. II 



78 



Mult. II 93 



O 

ERIC 

f 



4 

6 



Prob . 
Nos . 

29-31 



34 

7 36-37 

8 44-46 

43 2 6 

3 10-12 



2 8-11 



1 3 



9-11 

12 

14 

21-26 

32-34 



i' 



Comments 



Older., less bright students missing 
these. Reason obscure. 

Teaching materials have no practice 
for case of zero tens along with 
nonzero units . 

Revise test item. Nonconventional no. 
line representations not in teaching 
materials . 

Test format confusing; revise. 

Teaching materials lack emphasis on 
addition of zero. 

Problem format too difficult for 
younger and low ability children. 
Nothing similar in teaching materials. 

General evidence of unsuccessful 
achievement, but could be problem 
format on test. 

Sets of 1 not covered adequately in 
skill folder. 

Problem format may be too difficult. 
Add a worked example. 

Materials lacking example using one 
row or one column. 

Require specific response. 

Revise items to conform to materials 
Frovide intermediate steps. 

This skill folder and corresponding 
test items already revised. 

Test and related materials already 
revised. 



203 



Div. II 



91 



1 



3-8 






o 

ERIC 

>. 



Tos t 


No. of 
Cases 


1 Skill 


Prob. 
Nos . 


Div. II 


91 


2 


9-17 ' 






3 


18-24 






4 


27 


•Frac. II 


81 


1 


5 


M. Op. II 


84 


1 


4 






2 


8 


Meas . ' II 


109 


1 


9 



3 

4 

5 



Num. Ill 112 7 



10 

11 



20,21 

22,23 



24-30 

35 

36-39 



30-33 

34-40 



41-44 



45-48 

49-54 



Comments 



Test and related materials already 
revised 



Revise item to eliminate reading 
requirement. 



Move irregularly placed square into 
pattern . 



Materials need more emphasis on 
adding zero. 



Materials need more multiplication 
problems along with addition. 



Items already revised. 



Revise items so block divisions can 
be seen through shading. 



Frequently missed on posttest, yet 
very similar to checkup test. 
Reason obscure. 



Items already revised. 
Item already revised. 



Change format so all responses 
require writing a letter. Include 
item with bulky but light and small 
but heavy materials on scales. 



Change items using "smallest" 
rather than "least." 



Materials may need to be revised to 
provide more practice. Test items 
should be preceded by worked example. 



Same difficulty as Num. II, Skill 6. 
Older, less bright, students missing 
items. Worked example might help. 
Teachers may need to prescribe 
activities for older students. 



Same comments as for Skill 9. 



Teaching materials need more items 
requiring students to write rather 
than choose responses. 
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Tos t 




No. of 


•a 

a: 


[’.rob , 




Cases 


U] 


Nos . 


Add. 


III 


39 


2 


15 


Subt . 


III 


57 


1 


10, 11 








2 


18-21 


Mult. 


III 


79 


3 


25 


Div. 


III; 


89 


3 


21 








4 


22 


Frac. 


III 


159 


2 


19 








5 


41 


M. Op 


. Ill 


99 


2 


11 



35,40, 

41 



Common t s 

Students may need more practice with 
numbers, greater than ten at this point. 

Students may need more practice with 
problems of the form 10= -5. 

Too difficult for older less bright 
students. Reason obscure. 

Multiplication by one causing 
difficulty. Materials provide 
practice; therefore, reason obscure. 

No practice for two different division 
statements related to 5 x 2=10, viz. 
10+ 5=2 and 10 Hr 2=5. 

Too difficult for younger children. 
Worked example might help. 

More practice needed involving 
shading of non- contiguous areas. 

Test item already changed. 

Too difficult probably due to 
inadequate materials coverage of 
half dozens in Meas. II, Skill 4. 

Materials need more practice with 
items of the form 

0-4=16-5 and 



Time IT 



16-5=12- n . 

Insuffucient cases to analyze. 



Money III 



Insufficient cases to analyze. 



Meas. 


Ill 


94 


1 


4,5 


Num. 


IV 


142 


2 


19 








8 


47,49 


Add. 


IV 




2 


O 

I 

H* 

W 



14-20 



Yardstick needs to be redrawn. 

Materials do not teach counting 
backward. 

More practice needed in changing 
Roman numerals to Arabic (not the 
reverse) . 

Change problem format to that of 
check up test. 
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Problems should be preceded by 
worked example. 



Test 



• *f*rv* ***rr >v p 7 p ? t >^ *-■# 



k A™ V 



No. of 
Cases 



•H 

CO 



Subt. IV 


69 


Mult. IV 


124 


Div. IV 


105 



Frac.IV 1C5 



Prob. 

Nos. 

37 



Comments 

Two correct choices (item has 
been revised) . 



No deficiencies detected. 

1 4 Changes item to show divisor. 

2 17-19 Change to format of Item 16. 

(Questions do not conform to materials 

4 29 Replace item so that only a 

single response is required. 

2 9,12, Materials do not sufficiently 

14,17 cover case in which numerator is 

greater than 1 and nuirber of items 
in picture is 2, 3 or 4 times as 
large as denominator. 

5 37-40 Already changed to conform to 

materials. 

41-43 Should have worked example. 



M. Op. IV 


140 


1 


9-10 Eliminate double responses. 

Show worked example. 


Money IV 


72 


No 


deficiencies apparent. 


Time IV 


81 


5 


19-23 Problems generally too difficult. 

Revise to conform to checkup test 


Meas. IV 


110 


3 


12 Materials do not teach reading 

between marks on thermometer, but 
this is covered by objective. 


Num. V 


91 


No 


deficiencies apparent. 


Add. V 


86 


No 


deficiencies apparent. 


Subt. V 


82 


1 


13 LJ “4*4 too difficult. Reason 



obscure. 

44-48 Show worked example. Eliminate 
"Use . " 



50 



Students need more practice with 
regrouping minuends containing zeros. 



Mult. V 94 



No deficiencies apparent. 



o 
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Test 



Div. V 



No . of 
Cases 

103 



Frac. V 



M.Op. V 



Money V 
Time V 



> 



Meas. V 



Num. VI 



91 



82 



69 

72 



91 



119 



0 

ERIC 



M' 

CO 



Prob . 
Nos . 

21-24 



25 



36,41 



11,12 

1,6 

13 

14 



Comments 

Some tendency for brighter children 
to miss these. Format differs 
from checkup tests but not from 
some of practice pages. A worked 
example may hfelp. 

Frequently missed. This is only 
item which requires student to 
supply entire solution. Yet all 
checkup test items have this 
characteristic. Suggest more such 
items in format of checkup test. 

Omit items with remainder in 
testing this skill. 

Materials do not cover cases in 
which representations of fractions 
are of this type. 

Too difficult. Reason obscure. 

May be lack of practice with 
sixths and sevenths. 

Show spaces for conversion to cups 
or pints. Make reconversion to 
quarts a separate problem. 

Item inappropriate. 

Format confusing. 



No deficiencies apparent. 
2 7,11 



4 

3 



20 

16 

15 



32 



Clock minute hand points 
between hour marks difficult to 
read. More of this kind of problem. 

Same as above. 

Students probably confusing page 
width with test margins. Revise 
problem. 

End pdints of number line segment 
probably not prominently enough 
marked. Add more of this kind of 
problem. 

Students need more practice with 
numbers not discemable as prime 
or composite through knowledge 
of multiplication table, e.g., 51, 
57, 91, 121. 






Prob . 

Nos , Comments 

34,35 Students need more practice writing 
larger numbers as products of primes 
(These are on checkup tests.) 

38,39 Provide some test problems showing 
structure for answers, e.g., 

24= 2. 2*2*3 = 2* * 3. Then 
follow with problems in which 
students provide entire answer. 

42 Change to square of a prime, e.g. , 

49. 



Add. VI 


81 


No 


deficiencies apparent. 


Subt. VI 


77 


2 


13,15 


Difficulty when number of decimal 
places not same in subtrahend arid 
minuend. Materials need 
practice page devoted to this 
situation. 


Mult. VI 


79 


4 


17 


Probably needs a worked example. 


Div. VI 


70 


No 


deficiencies apparent. 


Prac. VI 


58 


2 


9-15 


6.C.F. not involved in teacning 
materials for this skill. Revise 
problems. 


M. Op. VI 


47 


No 


deficiencies apparent. 


Money VI 


51 


No 


deficiencies apparent. 


Time VI 


42 


5 


26 


Materials need to provide more 
practice for items of this kind. 


Meas.VI 


47 


5 


23-27 


Underline or use all caps for 



measurement units different from 
preceding ones in problem* 



Test 



No. of *h 
C ases </j 



Num. VI 



8 



m 



